Question

Difficulty: MediumStream Processing and Event Routing with Amazon Kinesis and EventBridge

A healthcare monitoring application ingests real-time vital signs from patient wearable devices into an Amazon Kinesis Data Stream. During peak clinical hours, the producer applications report frequent ProvisionedThroughputExceededException errors. Upon analysis, the developer finds that the aggregate write throughput is significantly below the stream's provisioned limit, but a subset of shards is heavily throttled. The application uses the device manufacturer name as the partition key.

Which action should the developer take to resolve the write throttling?

  1. A
    Update the IAM policy of the producer role to modify the trust relationship policy document.
  2. Change the partition key to a unique identifier such as the patient's device ID.Answer
  3. C
    Increase the execution timeout of the downstream AWS Lambda function that processes the Kinesis stream.
  4. D
    Deploy the downstream consumer AWS Lambda function in a private VPC subnet without a NAT Gateway.

Answer

Change the partition key to a unique identifier such as the patient's device ID.
Changing the partition key to a unique identifier, such as the patient's device ID, ensures high entropy. Kinesis uses the partition key hash value to determine which shard receives a data record. A high-cardinality key distributes the records evenly across all shards, resolving the hot shard problem and preventing write throttling.

Step-by-Step Solution

1
Analyze the cause of the ProvisionedThroughputExceededException.
Identify that a subset of shards is throttled while aggregate throughput is under limits, indicating a hot shard issue.
Throttling on specific shards while overall throughput is low points to uneven data distribution.
2
Examine the current partition key choice.
The current key is the device manufacturer name, which has low entropy (few unique values) and causes data to cluster onto a small number of shards.
A low-cardinality partition key leads to unbalanced shard allocation.
3
Select a high-entropy alternative partition key.
Choosing the patient's device ID provides a large number of unique values, distributing records uniformly across all shards.
High-entropy partition keys ensure even distribution of writes across all shards, eliminating hot shards.

Key Concept

Selecting an appropriate Kinesis partition key with high cardinality/entropy to avoid hot shards.
Estimated Time:1m 30s
Rate this question