Examkin

Question

Difficulty: MediumStream Processing and Event Routing with Amazon Kinesis and EventBridge

A developer is designing a real-time inventory management system for a global retail chain. Stock levels for millions of unique Stock Keeping Units (SKUs) across 500 warehouses are updated constantly. These updates are published to an Amazon Kinesis Data Stream. During peak sales events, the application's producers frequently encounter ProvisionedThroughputExceededException errors when writing to the stream, even though the overall write throughput of the stream is well below the stream's aggregate capacity. Which partition key design should the developer implement to resolve this issue?

  1. A
    Use the warehouse ID as the partition key to keep all events from the same location together in the same shard.
  2. Use the SKU as the partition key to distribute write operations evenly across all available shards.Answer
  3. C
    Use a static partition key such as 'INVENTORY_UPDATE' to guarantee strict global ordering of all updates.
  4. D
    Increase the execution timeout of the downstream AWS Lambda consumer function to allow it more time to process the throttled shard's backlog.

Answer

Use the SKU as the partition key to distribute write operations evenly across all available shards.
Using the SKU as the partition key provides high entropy (millions of unique values) compared to the number of shards. This ensures that records are distributed evenly across all shards, preventing any single shard from becoming a bottleneck (hot shard) and resolving the ProvisionedThroughputExceededException.

Step-by-Step Solution

1
Analyze the nature of the ProvisionedThroughputExceededException error during writes.
The error indicates that a single shard is exceeding its limits (1 MB/sec or 1,000 records/sec for writes), which is typically caused by an uneven distribution of records (a hot shard).
Understanding why the exception occurs is necessary to target the root cause (uneven record routing).
2
Evaluate the partition keys of the incoming records.
Using keys with low cardinality/entropy (like warehouse ID or static keys) causes Kinesis to hash multiple active records to the same shard.
Analyzing partition key entropy helps determine how evenly records will be hashed and mapped to the stream's shards.
3
Select a partition key with high entropy (cardinality).
Using the unique SKU (millions of values) ensures uniform hash distribution across all shards.
High-entropy partition keys ensure a balanced load across all shards, maximizing the aggregate throughput of the stream.

Key Concept

Selecting high-entropy partition keys is crucial in Amazon Kinesis Data Streams to prevent uneven data distribution and avoid hot shards that trigger ProvisionedThroughputExceededException during writes.
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic