Examkin

Question

Difficulty: MediumStream Processing and Event Routing with Amazon Kinesis and EventBridge

A developer is implementing a serverless stream processing application. Real-time telemetry data is ingested into an Amazon Kinesis data stream. An AWS Lambda function is configured with an active event source mapping to process the stream records. The Lambda function is deployed within a private subnet of an Amazon VPC to securely access an internal database, but it also needs to make HTTP calls to an external API to enrich the incoming telemetry data. During testing, the developer observes that the Kinesis stream is experiencing a hot shard issue, resulting in ProvisionedThroughputExceededException errors on a single shard even though overall stream throughput is well below the limits, and the Lambda function fails to connect to the external API, causing execution timeouts. Which set of modifications will resolve both issues?

  1. Modify the producer to use a high-entropy value such as a combination of device identifier and timestamp as the partition key, and deploy a NAT Gateway in a public subnet with routing configured from the private subnet.Answer
  2. B
    Modify the producer to use a static value such as the event status code as the partition key, and deploy a NAT Gateway in a public subnet with routing configured from the private subnet.
  3. C
    Modify the producer to use a high-entropy value such as a combination of device identifier and timestamp as the partition key, and attach an Internet Gateway directly to the private subnet containing the Lambda function.
  4. D
    Modify the producer to use a static value such as the event status code as the partition key, and increase the Lambda function execution timeout configuration to the maximum limit of 15 minutes.

Answer

Modify the producer to use a high-entropy value such as a combination of device identifier and timestamp as the partition key, and deploy a NAT Gateway in a public subnet with routing configured from the private subnet.
The correct answer resolves both issues by using a high-entropy partition key (device identifier and timestamp) to distribute stream ingestion traffic uniformly across shards, and by deploying a NAT Gateway in a public subnet to allow the Lambda function inside the private subnet to reach the external API.

Step-by-Step Solution

1
Address the hot shard issue in the Kinesis stream by using a high-entropy partition key.
Selecting a partition key that has high entropy (such as a device identifier combined with a timestamp) ensures that records are distributed uniformly across all available shards, avoiding ProvisionedThroughputExceededException errors.
Kinesis determines which shard a record is sent to based on the hash of its partition key. Low-entropy keys cause data to concentrate on specific shards.
2
Address the Lambda network connectivity issue in the VPC.
Deploy a NAT Gateway in a public subnet of the VPC and update the private subnet's route table to forward 0.0.0.0/0 traffic to the NAT Gateway.
Lambda functions in private VPC subnets do not have public IP addresses and cannot communicate directly with the public internet via an Internet Gateway. A NAT Gateway is required to translate private IPs to public IPs for outbound traffic.

Key Concept

Partition key design in Amazon Kinesis Data Streams and egress network routing for AWS Lambda functions in a VPC.
Estimated Time:1m 30s
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic