Examkin

Question

Difficulty: HardStream Processing and Event Routing with Amazon Kinesis and EventBridge

A developer is implementing a retail order processing pipeline. A producer application writes order updates to an Amazon Kinesis data stream using the customer's country code as the partition key. An AWS Lambda function, configured within a private VPC subnet without internet access, processes the stream and must route orders exceeding 10,00010,000 USD to an external system via an Amazon EventBridge custom event bus.

During high-traffic events, the developer observes two issues:
1. The Kinesis stream experiences `ProvisionedThroughputExceededException` errors on a single shard, even though overall stream throughput is well below the stream limits.
2. The Lambda function times out and fails to publish the filtered high-value events to the EventBridge event bus.

Which combination of actions will resolve both issues?

  1. Change the Kinesis producer to use a high-entropy value such as the unique order ID as the partition key, and create an interface VPC endpoint (AWS PrivateLink) for EventBridge in the Lambda function's VPC subnets.Answer
  2. B
    Increase the number of shards in the Kinesis stream to distribute the load, and assign a public IP address to the Lambda function's ENIs to allow it to communicate with EventBridge directly over the internet.
  3. C
    Update the Kinesis producer to use a static, constant string as the partition key to ensure order preservation, and update the trust policy of the Lambda execution role to allow the EventBridge service to publish events.
  4. D
    Keep the country code partition key but increase the Lambda batch size to process more records per invocation, and increase the Lambda function's timeout limit to allow the network calls to EventBridge to complete.

Answer

Change the Kinesis producer to use a high-entropy value such as the unique order ID as the partition key, and create an interface VPC endpoint (AWS PrivateLink) for EventBridge in the Lambda function's VPC subnets.
The correct answer resolves both the database streaming bottleneck and the VPC connectivity bottleneck. By using a high-entropy key like a unique order ID, records are distributed uniformly across all Kinesis shards, resolving the hot shard throttling. By deploying an interface VPC endpoint (PrivateLink) for EventBridge inside the VPC, the Lambda function can route its API calls locally to the endpoint, enabling event routing without requiring public internet routing.

Step-by-Step Solution

1
Analyze the Kinesis throttling root cause.
The producer uses the country code as the partition key. Because country codes have low cardinality/entropy, some countries with high order volumes will flood a single shard, causing a 'hot shard' and resulting in ProvisionedThroughputExceededException errors even if the overall stream capacity is sufficient.
Kinesis routes records to shards based on the hash of the partition key; high-entropy keys ensure uniform distribution.
2
Select a proper partition key strategy.
Change the partition key to a high-entropy identifier like a unique order ID to distribute writes evenly across all available shards.
This fixes the uneven write distribution and prevents the ProvisionedThroughputExceededException on individual shards.
3
Identify the Lambda connectivity issue to EventBridge.
The Lambda function is running in a private VPC subnet with no internet access. Since EventBridge endpoints are public by default, the Lambda function cannot resolve or reach the EventBridge service, causing the function to time out.
VPC-enabled Lambda functions without a NAT Gateway or VPC endpoint cannot access public AWS service endpoints.
4
Resolve the VPC network connectivity.
Create an interface VPC endpoint (AWS PrivateLink) for EventBridge (com.amazonaws.region.events) inside the Lambda function's VPC.
This establishes private connectivity, allowing the Lambda function to publish events to EventBridge securely without traversing the public internet.

Key Concept

Kinesis partition key entropy design and VPC Lambda networking with EventBridge PrivateLink.
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic