Examkin

Question

Difficulty: EasyStream Processing and Event Routing with Amazon Kinesis and EventBridge

A developer is building a high-volume clickstream analytics pipeline where web applications publish event data to Amazon Kinesis Data Streams. The stream is configured with 4 shards, but the developer notices ProvisionedThroughputExceededException errors during high traffic. Upon review, they find that the partition key is set to a constant string value 'clickstream_event'. Which two actions should the developer take to resolve the throttling errors and optimize stream throughput?

  1. Modify the producer application to use the unique session ID or user ID as the partition key for each record.Answer
  2. Increase the shard count of the Kinesis Data Stream to scale the write throughput capacity.Answer
  3. C
    Modify the producer application to use a static value like the application environment name as the partition key.
  4. D
    Deploy the consumer Lambda function in a private VPC subnet without configuring a NAT Gateway or VPC endpoint.
  5. E
    Increase the Lambda function execution timeout to its maximum limit to allow more processing time.

Answer

Modify the producer application to use the unique session ID or user ID as the partition key, and increase the shard count of the Kinesis Data Stream.
To resolve the throttling issues caused by a hot shard, the developer must select a partition key with high entropy (such as a unique session ID or user ID) so that data is distributed evenly across all shards. Additionally, if the overall volume exceeds the total capacity of the stream, increasing the shard count provides more aggregate throughput capacity.

Step-by-Step Solution

1
Analyze the cause of the ProvisionedThroughputExceededException error.
The error occurs because a static partition key routes all traffic to a single shard, which exhausts its capacity.
Kinesis determines shard routing by hashing the partition key. A constant key sends all records to the same shard.
2
Change the partition key design to use a high-entropy attribute.
Using attributes like user ID or session ID distributes the records evenly across all available shards.
This resolves the hot shard problem by spreading the throughput load across the stream.
3
Scale the Kinesis Data Stream capacity.
Increasing the shard count increases the total write capacity of the stream.
Each shard provides up to 1 MB/sec or 1,000 records/sec write capacity, so scaling the shard count matches high traffic demands.

Key Concept

Partition key design and shard capacity scaling in Amazon Kinesis Data Streams
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic