An advertising technology company is designing a high-performing ingestion and analytics p… — AWS Certified Solutions Architect - Associate

An advertising technology company is designing a high-performing ingestion and analytics platform to process user interaction events from mobile devices globally. The platform receives a peak traffic rate of $120,000$ events per second, with each JSON event payload averaging $1.2\text{ KB}$ in size. The system must meet the following requirements:

* Deliver all raw events to Amazon S3 in Apache Parquet format for historical analytics, with a maximum acceptable delivery latency of 5 minutes.
* Index enriched events into Amazon OpenSearch Service for real-time dashboard analytics with sub-second latency.
* Scale dynamically or be pre-provisioned to handle sudden traffic spikes of up to $1.5$ times the peak traffic rate without throttling or data loss.

Which combination of steps should the solutions architect take to meet these requirements with the highest performance and reliability? (Select two.)

Configure an Amazon Kinesis Data Stream in Provisioned mode with 220 shards to ingest the incoming event stream.Answer
Configure Amazon Data Firehose to consume from the Kinesis Data Stream, buffer and convert the records to Apache Parquet format using an AWS Glue schema, and deliver them to Amazon S3; in parallel, configure an AWS Lambda function to consume from the same stream and write to Amazon OpenSearch Service.Answer
C
Configure an Amazon Kinesis Data Stream in Provisioned mode with 120 shards to handle the peak write throughput of 144 MB/s.
D
Ingest the events into an Amazon SQS Standard queue to decouple the ingestion layer, and configure an AWS Lambda function to poll the queue and write events to Amazon OpenSearch Service in strict chronological order.
E
Direct the mobile devices to write events directly to Amazon Data Firehose, using its built-in data transformation with AWS Lambda to convert the data to Parquet before writing to both Amazon S3 and Amazon OpenSearch Service.

Answer

The correct combination is to provision an Amazon Kinesis Data Stream with 220 shards to ingest the incoming event stream, and use Amazon Data Firehose in parallel with AWS Lambda as consumers of the stream to deliver data to Amazon S3 and Amazon OpenSearch Service respectively.

Provisioning 220 shards on the Amazon Kinesis Data Stream accommodates the

216\text{ MB/s}

throughput and

180,000

events/sec spike requirements. Connecting Amazon Data Firehose for S3 delivery and AWS Lambda for OpenSearch delivery in parallel allows the system to support different processing patterns: Firehose handles structured batch ingestion and conversion to Parquet, whereas Lambda achieves the sub-second response times required for OpenSearch dashboards.

Step-by-Step Solution

Calculate the peak and spike record rates and write throughput requirements.

Peak rate is

120,000

records/sec at

144\text{ MB/s}

(

120,000 \times 1.2\text{ KB}

). The spike rate (1.5x) is

180,000

records/sec at

216\text{ MB/s}

(

180,000 \times 1.2\text{ KB}

Sizing the ingestion layer requires calculating both the records per second and the MB/s payload size at the maximum expected spike rate.

Determine the required number of Kinesis Data Streams shards.

Each shard supports

1,000

records/sec or

1\text{ MB/s}

write throughput. To support

180,000

records/sec, we need at least

180

shards. To support

216\text{ MB/s}

throughput, we need at least

216

shards. We select the option with 220 shards.

Provisioning must meet the highest requirement from both records/sec and MB/s dimensions to prevent ingestion bottlenecks.

Choose the high-performing ingestion consumer architecture for multi-destination delivery.

Configure Amazon Data Firehose as a consumer to convert records to Parquet via AWS Glue and deliver to S3 within 5 minutes. Set up AWS Lambda to consume from the same stream and push directly to Amazon OpenSearch Service.

Firehose handles micro-batching and schema conversion to Parquet efficiently for S3, while AWS Lambda processes stream events immediately to achieve sub-second latency for OpenSearch.

Key Concept

Calculating Kinesis Data Streams shard capacity based on both records per second and payload throughput limits, and implementing parallel consumer patterns for diverging latency requirements.

Estimated Time:3m 0s

Question

Answer

Step-by-Step Solution

Key Concept

Common Mistakes

Topics

More questions from this topic