Examkin

Question

Difficulty: HardStream Processing and Event Routing with Amazon Kinesis and EventBridge

A developer is implementing a real-time tracking pipeline for a logistics platform. A producer application writes delivery location updates to an Amazon Kinesis Data Stream using the delivery region name (e.g., 'us-east-1') as the partition key. A downstream AWS Lambda function is configured to process the stream via an event source mapping and must call an external third-party mapping API to validate coordinates. The Lambda function is deployed within private subnets of a VPC. During peak hours, the developer observes ProvisionedThroughputExceededException errors on the Kinesis stream, despite the total data volume being well below the stream's aggregate limit. Additionally, the Lambda function fails to connect to the external API. Which combination of changes should the developer implement to resolve both the Kinesis throttling and the connection issues?

  1. A
    Increase the Kinesis stream shard count to handle peak load. Move the Lambda function to a public subnet to allow direct access to the internet.
  2. Update the producer application to use a high-entropy key, such as a unique delivery ID, as the partition key. Configure a NAT Gateway in a public subnet and route internet-bound traffic from the Lambda function's private subnet through it.Answer
  3. C
    Update the producer application to use a high-entropy key, such as a unique delivery ID, as the partition key. Configure an AWS PrivateLink VPC endpoint for the external third-party API.
  4. D
    Keep the delivery region name as the partition key to preserve ordering, and increase the Kinesis stream shard count. Configure a NAT Gateway in a public subnet and route internet-bound traffic from the Lambda function's private subnet through it.

Answer

Update the producer application to use a high-entropy key, such as a unique delivery ID, as the partition key. Configure a NAT Gateway in a public subnet and route internet-bound traffic from the Lambda function's private subnet through it.
The correct answer correctly identifies that a high-entropy key like a unique delivery ID is required to distribute writes evenly across all shards, resolving the ProvisionedThroughputExceededException throttling. It also correctly specifies that a NAT Gateway in a public subnet is required to enable outbound internet connectivity for the VPC-enabled Lambda function so that it can reach the third-party API.

Step-by-Step Solution

1
Analyze the cause of ProvisionedThroughputExceededException.
The current partition key is the region name, which has very low entropy. During peak hours, a large volume of writes goes to the same region, directing all traffic to a single shard (hot shard) and causing write throttling.
Kinesis routes records to shards based on the hash of the partition key; low entropy partition keys lead to uneven distribution.
2
Select a proper partition key strategy.
Using a unique delivery ID as the partition key provides high entropy, distributing writes evenly across all available shards.
High-entropy keys ensure balanced utilization of stream throughput capacity.
3
Analyze the Lambda function's connectivity issue.
The Lambda function is in private subnets and lacks internet connectivity to call the external third-party API.
VPC-enabled Lambda functions require a NAT Gateway configured in a public subnet with routing rules on the private subnet to connect to the public internet.

Key Concept

Handling Kinesis Data Stream hot shards using high-entropy partition keys, and configuring outbound internet access for Lambda functions running inside a VPC.
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic