Examkin

Question

Difficulty: MediumStream Processing and Event Routing with Amazon Kinesis and EventBridge

A developer is building a smart home application that processes real-time device state updates from thousands of smart hubs. The hubs publish state updates to an Amazon Kinesis Data Stream. An AWS Lambda function, configured with an Event Source Mapping, processes the stream in batches. The Lambda function needs to query an Amazon ElastiCache for Redis cluster located in a private VPC subnet to retrieve device owner metadata, and then send alert notifications to an external push notification gateway API.

The developer configures the Lambda function to run inside the same private VPC subnets. However, during testing, the Lambda function fails to connect to the external API, resulting in batch processing failures.

Which configuration should the developer implement to resolve the connectivity issue and process the stream efficiently?

  1. A
    Keep the Lambda function in the private subnets without a NAT Gateway, and increase the Lambda function execution timeout to the maximum limit of 1515 minutes to allow network connection retries to eventually succeed when the container is reused.
  2. Provision a NAT Gateway in a public VPC subnet, configure the route table of the private subnets to direct outbound traffic (0.0.0.0/00.0.0.0/0) to the NAT Gateway, and ensure the smart hubs write to the Kinesis stream using a high-entropy partition key, such as a unique device ID.Answer
  3. C
    Modify the Lambda function's IAM trust policy to trust the VPC service (`vpc.amazonaws.com`), and configure the smart hubs to use a static string as the partition key to ensure all events are processed by a single shard in chronological order.
  4. D
    Move the Lambda function to a public VPC subnet with direct internet access, and configure the smart hubs to use a low-entropy partition key, such as the device manufacturer name, to reduce the complexity of the stream structure.

Answer

Provision a NAT Gateway in a public VPC subnet, configure the route table of the private subnets to direct outbound traffic to the NAT Gateway, and ensure the smart hubs write to the Kinesis stream using a high-entropy partition key, such as a unique device ID.
To allow the Lambda function to reach the external notification gateway API while retaining access to the private ElastiCache cluster, the function must remain in private subnets, and internet-bound traffic (0.0.0.0/00.0.0.0/0) must be routed through a NAT Gateway provisioned in a public subnet. Additionally, using a high-entropy partition key (such as the unique device ID) is critical for stream efficiency as it distributes the incoming payloads evenly across shards, preventing the Kinesis stream from experiencing hot shards and ProvisionedThroughputExceededException.

Step-by-Step Solution

1
Analyze the network configuration of the Lambda function running inside the VPC.
The Lambda function is inside private subnets and can access the ElastiCache cluster, but it cannot access the external API because private subnets do not have direct routes to the internet.
Identify why the Lambda function fails to connect to the external API.
2
Determine the required network components for outbound internet access from private VPC subnets.
A NAT Gateway must be provisioned in a public subnet, and the route table for the private subnets must direct outbound traffic (0.0.0.0/00.0.0.0/0) to the NAT Gateway.
Establish routing for outbound external API requests from private subnets.
3
Evaluate Kinesis partition key strategies for processing updates from thousands of devices.
A high-entropy partition key (like a unique device ID) distributes records evenly across shards, avoiding hot shards and provisioning issues, whereas static or low-entropy keys (like manufacturer name) lead to hot shards.
Optimize stream processing performance and avoid write throttling.

Key Concept

Outbound VPC connectivity for AWS Lambda and partition key optimization for Kinesis Data Streams.
Rate this question

Topics

Stream Processing and Event Routing with Amazon Kinesis and EventBridge

More questions from this topic