Soru

Zorluk: OrtaHigh-Performing Data Ingestion and Transformation Solutions

A gaming company is launching a new multiplayer game and needs to design a high-performing data ingestion and transformation pipeline for player session telemetry. The pipeline must ingest real-time telemetry events and process them in strict chronological order per player session to update live leaderboards with sub-second latency. Concurrently, the same raw event data must be converted into Apache Parquet format and stored in Amazon S3 for long-term historical analytics. Which combination of AWS services should a solutions architect select to meet these requirements? (Select two.)

  1. A
    Ingest the player telemetry into a standard Amazon SQS queue to decouple the pipeline, and configure an AWS Lambda function to poll the queue and update the live leaderboards.
  2. B
    Configure Amazon Kinesis Data Streams in provisioned mode to ingest the player telemetry, and rely on the stream to automatically split shards during peak gaming events when throughput limits exceed 1 MB/s1\text{ MB/s} or 1,0001,000 records per second per shard.
  3. C
    Use Amazon Data Firehose to ingest the telemetry, convert it to Parquet, and write the files to Amazon S3 Standard-Infrequent Access (S3 Standard-IA) with a lifecycle policy to delete the files after 1414 days.
  4. Ingest the telemetry using Amazon Kinesis Data Streams, specifying the player session ID as the partition key, and consume the stream using a custom application to update the leaderboards.Cevap
  5. Use Amazon Data Firehose to read from the Kinesis Data Stream, use AWS Glue to transform the telemetry data to Parquet format, and write the output files to Amazon S3.Cevap

Cevap

To ingest and transform player session telemetry while maintaining strict chronological ordering per session with sub-second latency, use Amazon Kinesis Data Streams partitioned by the player session ID for the real-time path, and consume from the same stream using Amazon Data Firehose with AWS Glue to convert the telemetry to Parquet format for Amazon S3 storage.
The correct answer consists of using Amazon Kinesis Data Streams for real-time ingestion, combined with Amazon Data Firehose and AWS Glue for near-real-time transformation and storage. Ingesting telemetry into Amazon Kinesis Data Streams with the player session ID as the partition key guarantees that all records for a given session are routed to the same shard and processed in strict order. A custom consumer application can read this stream to update the leaderboard with sub-second latency. Concurrently, Amazon Data Firehose can consume the same Kinesis stream, invoke AWS Glue for schema-based transformation to Parquet format, and deliver the structured files to Amazon S3, meeting the analytics requirement without administrative overhead.

Adım Adım Çözüm

1
Analyze ingestion and ordering requirements
Identify that player telemetry requires sub-second processing and strict chronological ordering per session ID.
Guaranteed ordering per session is necessary to prevent out-of-order state updates on leaderboards.
2
Evaluate real-time ingestion options for ordered data
Select Amazon Kinesis Data Streams with the session ID as the partition key.
Kinesis Data Streams routes records with the same partition key to the same shard, preserving the ingestion sequence. Custom consumers can then process shards with sub-second latency.
3
Evaluate transformation and storage requirements
Configure Amazon Data Firehose to read from the Kinesis stream, use AWS Glue for Parquet conversion, and write to Amazon S3.
This provides a serverless, zero-maintenance mechanism to batch, transform, and store historical data in a high-performing columnar format (Parquet).

Anahtar Kavram

Decoupling real-time stream ingestion and near-real-time batch transformation using Amazon Kinesis Data Streams and Amazon Data Firehose to meet distinct performance, ordering, and storage format constraints.
Tahmini Süre:2m 0s
Bu soruyu puanla