Examkin

Soru

Zorluk: OrtaHigh-Performing Data Ingestion and Transformation Solutions

A media company is building a telemetry platform to ingest and transform clickstream data from its mobile application. The data stream averages 8 MB/s8\text{ MB/s} and spikes up to 25 MB/s25\text{ MB/s} during peak hours. The company requires that the data be converted from JSON to Apache Parquet format, partitioned by date, and stored in Amazon S3. The latency from ingestion to S3 must not exceed 5 minutes (300 seconds300\text{ seconds}). The architecture must be high-performing and require minimal operational overhead. Which two actions should a solutions architect recommend to meet these requirements?

  1. Create an Amazon Kinesis Data Firehose delivery stream to ingest the data, and enable data format conversion to Apache Parquet using the AWS Glue Data Catalog.Cevap
  2. Configure the Kinesis Data Firehose delivery stream with a buffer interval of 300 seconds and direct the output to the destination Amazon S3 bucket.Cevap
  3. C
    Deploy an Amazon Kinesis Data Stream with a single provisioned shard to ingest the stream sequentially before routing it to the S3 bucket.
  4. D
    Ingest the clickstream data into an Amazon SQS Standard queue to decouple the ingestion tier and guarantee the chronological order of user events.
  5. E
    Store the incoming unconverted clickstream data in an Amazon S3 bucket using the S3 Standard-Infrequent Access (S3 Standard-IA) storage class to optimize storage costs before a weekly batch conversion job.

Cevap

Create an Amazon Kinesis Data Firehose delivery stream to ingest the data and configure it to perform format conversion to Apache Parquet using AWS Glue, then set the buffer interval to 300 seconds to write the files to the target Amazon S3 bucket.
The correct solution involves creating an Amazon Kinesis Data Firehose delivery stream configured to perform inline conversion of incoming JSON data to Apache Parquet format using the AWS Glue Data Catalog. By setting the buffer interval to 300 seconds, the delivery stream ensures the transformed data is written to the destination S3 bucket within the required 5-minute latency window without manual operational management.

Adım Adım Çözüm

1
Select an ingestion service that supports automatic, serverless data format conversion.
Amazon Kinesis Data Firehose is selected because it can convert JSON payloads to Apache Parquet format on the fly using schemas from the AWS Glue Data Catalog, avoiding custom translation code or servers.
This minimizes operational overhead while maintaining high performance.
2
Configure the buffer settings to meet the latency requirement.
The buffer interval is set to 300 seconds (5 minutes).
This guarantees that data is flushed and written to S3 within the required 5-minute time window.
3
Evaluate and reject options that introduce throttling risks or lack ordering/cost efficiency.
Reject the options involving a single Kinesis shard (which throttles at 1 MB/s1\text{ MB/s}), SQS Standard (which does not guarantee ordering), and S3 Standard-IA (which charges a 30-day minimum duration penalty).
This ensures the final solution meets throughput, ordering, and cost constraints.

Anahtar Kavram

Serverless data ingestion and real-time transformation using Kinesis Data Firehose and AWS Glue.
Tahmini Süre:2m 0s
Bu soruyu puanla

Konular

High-Performing Data Ingestion and Transformation Solutions

Bu konudan daha fazla soru