A logistics company is designing an ingestion system for real-time telemetry from 50,000 d… — AWS Certified Solutions Architect - Associate

A logistics company is designing an ingestion system for real-time telemetry from $50,000$ delivery vehicles. Each vehicle sends a $2\text{ KB}$ JSON payload every $5\text{ seconds}$ containing GPS coordinates and sensor status. The company needs to ingest this data, transform it into Apache Parquet format, partition it by vehicle type and date, and load it into Amazon S3 for historical analysis. The system must scale automatically to handle traffic spikes, and the end-to-end data latency must be under $15\text{ minutes}$ . Which combination of actions should the solutions architect take to meet these requirements with the least operational effort? (Select two.)

Create an Amazon Kinesis Data Stream in On-Demand capacity mode to ingest the incoming vehicle telemetry.Cevap
Create an Amazon Data Firehose delivery stream configured with the Kinesis Data Stream as the source, enabling record format conversion to Apache Parquet using AWS Glue.Cevap
C
Create an Amazon Kinesis Data Stream in Provisioned capacity mode with a fixed shard count of $5$ shards to ingest the incoming telemetry.
D
Create an Amazon SQS standard queue to receive the telemetry data directly from the vehicles, and configure an AWS Glue ETL job to poll the queue continuously.
E
Create an Amazon SQS FIFO queue with message deduplication to store the records, and use Amazon EventBridge pipes to transform the messages and write them to S3.

Cevap

The correct actions are to create an Amazon Kinesis Data Stream in On-Demand capacity mode to ingest the vehicle telemetry and create an Amazon Data Firehose delivery stream with the Kinesis Data Stream as the source, enabling record format conversion to Apache Parquet using AWS Glue.

An Amazon Kinesis Data Stream in On-Demand capacity mode handles the high peak ingestion throughput of

20\text{ MB/s}

and

10,000\text{ records/s}

automatically. Using Amazon Data Firehose to read from the stream and transform the JSON data into Parquet using AWS Glue provides a serverless, low-overhead solution that meets the latency target.

Adım Adım Çözüm

Calculate the peak ingestion throughput and record rate from the vehicle fleet telemetry.

Peak throughput is calculated as

\frac{50,000 \text{ vehicles} \times 2 \text{ KB}}{5 \text{ seconds}} = 20 \text{ MB/s}

, and peak message rate is

\frac{50,000 \text{ records}}{5 \text{ seconds}} = 10,000 \text{ records/s}

This establishes the performance limits that any selected AWS ingestion service must support without throttling.

Evaluate Kinesis Data Streams shard capacity modes against the calculated requirements.

A provisioned stream with 5 shards supports only

5 \text{ MB/s}

and

5,000 \text{ records/s}

, which is insufficient. An On-Demand stream automatically scales to handle up to

200 \text{ MB/s}

and

200,000 \text{ records/s}

, comfortably meeting the peak.

Choosing Kinesis Data Stream in On-Demand mode ensures write throughput capacity scales automatically to handle traffic spikes.

Select a serverless, low-overhead transformation and delivery mechanism to load data into S3.

Amazon Data Firehose integrates directly with Kinesis Data Streams as a source, performs inline format conversion to Apache Parquet using AWS Glue schema definitions, and writes the output directly to S3 within the 15-minute latency requirement.

This minimizes operational overhead by avoiding custom EC2 or Lambda code for formatting and buffering.

Anahtar Kavram

Serverless high-throughput real-time stream ingestion and transformation using Amazon Kinesis Data Streams and Amazon Data Firehose.

Soru

Cevap

Adım Adım Çözüm

Anahtar Kavram

Sık Yapılan Hatalar

Konular

Bu konudan daha fazla soru