Examkin

Soru

Zorluk: OrtaHigh-Performing Data Ingestion and Transformation Solutions

A logistics company monitors 5,000 container tracking devices that send temperature, humidity, and location telemetry every minute. The company wants to ingest, transform the coordinates into a standardized format, and load the data into Amazon Redshift in near-real-time. The telemetry must be processed in chronological order per container to ensure accurate tracking. The system must scale automatically to handle traffic spikes during peak shipping seasons and require minimal operational overhead. Which solution meets these requirements?

  1. Configure Amazon Kinesis Data Streams in On-Demand mode to ingest the telemetry using the container ID as the partition key. Use Amazon Data Firehose to read from the stream, invoke an AWS Lambda function to transform the coordinates, and deliver the data to Amazon Redshift.Cevap
  2. B
    Configure Amazon Kinesis Data Streams in Provisioned mode with a fixed number of shards to ingest the telemetry. Set up an Amazon EC2 Auto Scaling group to consume the streams, transform the coordinates, and copy the data to Amazon Redshift.
  3. C
    Ingest the telemetry data using an Amazon SQS standard queue, trigger an AWS Lambda function to transform the coordinates, and insert the records into Amazon Redshift.
  4. D
    Set up an Amazon Managed Streaming for Apache Kafka (Amazon MSK) cluster to ingest the telemetry, and run a scheduled AWS Glue ETL job to transform and load the data into Amazon Redshift.

Cevap

Configure Amazon Kinesis Data Streams in On-Demand capacity mode with the container ID as the partition key, and use Amazon Data Firehose integrated with AWS Lambda to transform and deliver the data directly to Amazon Redshift.
The correct answer provides a fully serverless, highly performant architecture. Setting Amazon Kinesis Data Streams to On-Demand capacity mode satisfies the scaling requirement without manual intervention. Partitioning the stream by container ID ensures strict chronological ordering per device. Amazon Data Firehose integrates natively with Kinesis Data Streams and AWS Lambda, allowing coordinates to be transformed in near-real-time before delivery to Amazon Redshift. This keeps operational overhead to a minimum.

Adım Adım Çözüm

1
Ensure ordered processing by container.
Using the container ID as the partition key in Amazon Kinesis Data Streams ensures that all telemetry payloads for a specific container are mapped to the same shard, which preserves their relative chronological order during ingestion.
Data within a single shard of a Kinesis Data Stream is ordered sequentially.
2
Select an auto-scaling, serverless ingestion layer.
Configuring Kinesis Data Streams in On-Demand mode allows the ingestion throughput to dynamically scale up or down based on incoming volume without manually managing shards.
This meets the requirement of handling traffic spikes automatically while minimizing operational overhead.
3
Integrate with a serverless delivery and transformation pipeline.
Connecting Amazon Data Firehose to the Kinesis stream enables serverless buffering and delivery to Amazon Redshift. Inlining an AWS Lambda function in the Firehose configuration allows data transformation (such as coordinate flattening) to occur during transit.
This workflow avoids the need to build, maintain, and scale custom consumer applications on EC2.

Anahtar Kavram

Serverless, ordered data ingestion and transformation using Amazon Kinesis Data Streams (On-Demand), Amazon Data Firehose, and AWS Lambda.
Tahmini Süre:2m 0s
Bu soruyu puanla

Konular

High-Performing Data Ingestion and Transformation Solutions

Bu konudan daha fazla soru