Examkin

Question

Difficulty: MediumStream Processing and Event Routing with Amazon Kinesis and EventBridge

A collaborative document editing platform processes real-time change events using an Amazon Kinesis Data Stream with 8 shards. The event payload includes `document_id` (a UUID), `user_id` (a UUID), `event_type` (e.g., `edit_text`, `update_style`, `document_deleted`), and `payload_size`. Currently, the producer application uses `event_type` as the partition key. During peak usage hours, write operations to the stream frequently fail with a `ProvisionedThroughputExceededException` even though the overall data rate is well below the stream's total limit. Additionally, the development team needs to route only the `document_deleted` events to an administrative Amazon SNS topic for compliance auditing. Which TWO actions should the developer take to resolve the throttling issue and route the compliance events?

  1. Change the partition key on the producer to use the `document_id` instead of the `event_type`.Answer
  2. Configure an Amazon EventBridge pipe with the Kinesis Data Stream as the source, specify a filter pattern matching the `document_deleted` event type, and set the Amazon SNS topic as the target.Answer
  3. C
    Change the partition key on the producer to a static string like `document_event_stream` to force all write operations into a single shard.
  4. D
    Deploy a consumer Lambda function inside a private VPC subnet without a NAT Gateway or VPC Endpoints for Amazon SNS to process the stream and publish to the SNS topic.
  5. E
    Write a consumer Lambda function with a 1-second timeout that processes the stream and initializes a new database connection inside the handler function on every invocation.

Answer

Change the partition key on the producer to use the document ID instead of the event type, and configure an Amazon EventBridge pipe with the Kinesis Data Stream as the source, defining a filter pattern for the deleted event type with the Amazon SNS topic as the target.
Using the document identifier as the partition key ensures high entropy and distributes write requests evenly across all shards, resolving the ProvisionedThroughputExceededException throttling caused by the hot shards when using a low-entropy key like the event type. Amazon EventBridge Pipes provides a direct, serverless way to consume records from a Kinesis Data Stream, filter them based on the event payload (specifically matching the deleted event type), and route them to targets like Amazon SNS without the need to write and manage custom consumer code.

Step-by-Step Solution

1
Analyze the cause of Kinesis write throttling.
The current partition key is the event type, which has low cardinality (few unique values). This results in uneven data distribution across shards, creating 'hot shards' that throttle writes.
Identifying that the low-entropy partition key is causing the ProvisionedThroughputExceededException.
2
Select a high-entropy partition key.
Using the document ID as the partition key distributes write operations uniformly across all shards while preserving ordering for updates to the same document.
A partition key with high cardinality ensures balanced shard utilization and prevents hot shards.
3
Determine the routing mechanism for compliance events.
Use Amazon EventBridge Pipes to consume events directly from the Kinesis stream, apply a filter pattern to match only the deleted events, and route them to the SNS topic.
EventBridge Pipes offers a low-latency, codeless integration pattern to filter and route stream records to AWS targets.

Key Concept

Selecting high-entropy partition keys for Kinesis Data Streams and filtering event streams using Amazon EventBridge Pipes.
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic