Examkin

Question

Difficulty: Very hardStream Processing and Event Routing with Amazon Kinesis and EventBridge

A retail company processes high-frequency transactional data using an Amazon Kinesis Data Stream with 55 shards. An AWS Lambda function acts as the consumer to process and save these records to Amazon DynamoDB. The records contain a `transactionId` (UUID), `customerId` (high entropy), `region` (only 55 unique values), and `transactionType` (e.g., 'purchase', 'refund'). The developer notices frequent `ProvisionedThroughputExceededException` errors on one specific shard, while the other shards are underutilized. Additionally, the developer needs to route all 'refund' transactions with an amount greater than $1000\$1000 to a third-party audit API using Amazon EventBridge.

Which two configurations should the developer implement to resolve the throttling issue and route the required transaction events?

  1. Configure the Kinesis producer to use `customerId` or `transactionId` as the partition key, and implement exponential backoff with jitter on write failures.Answer
  2. Create an EventBridge rule on the default event bus with an event pattern filtering for `transactionType` matching 'refund' and `amount` greater than 10001000, and set the target as an EventBridge API Destination.Answer
  3. C
    Configure the producer to use the `region` attribute as the partition key for the Kinesis Data Stream.
  4. D
    Increase the Lambda function's maximum execution timeout to 1515 minutes and set the batch size to 1000010000.
  5. E
    Deploy the consumer Lambda function in a private VPC subnet without a NAT Gateway or VPC endpoints.

Answer

To resolve the throttling issue, configure the Kinesis producer to use a high-entropy key like customerId or transactionId for even distribution and use exponential backoff on writes. To route the high-value refunds, create an EventBridge rule with the appropriate event pattern targeting an EventBridge API Destination.
The throttling issue is resolved by utilizing high-cardinality partition keys like customerId or transactionId, which spreads requests evenly across all shards. Implementing exponential backoff with jitter helps manage transient peaks. The event routing is resolved by configuring an EventBridge rule matching the specific JSON criteria and routing the matching events to the third-party API through an EventBridge API Destination, which manages the API integration natively.

Step-by-Step Solution

1
Analyze partition key cardinality
Using geographic region (low cardinality) leads to concentrated throughput on specific shards (hot shards). Select transactionId or customerId for high cardinality.
Kinesis distributes data to shards based on the MD5 hash of the partition key; higher entropy ensures uniform distribution.
2
Configure write retry policy
Incorporate exponential backoff with jitter on the producer side.
This mitigates transient provisioning failures without overwhelming the stream during spikes.
3
Set up EventBridge event routing
Define an EventBridge rule that filters transaction events based on type and amount.
Allows declarative routing of matching event payloads without writing custom sorting logic in Lambda.
4
Configure target endpoint routing
Define an API Destination for EventBridge pointing to the third-party endpoint.
API Destinations manage authorization, rate limits, and invocation mechanisms out-of-the-box.

Key Concept

Partition key design in Amazon Kinesis Data Streams and EventBridge routing patterns with API Destinations

Alternative Method

Instead of custom consumer code in Lambda, EventBridge Pipes could be configured to read directly from the Kinesis stream and route filtered events to the API Destination.
Estimated Time:3m 0s
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic