Examkin

Question

Difficulty: MediumHigh-Performing Data Ingestion and Transformation Solutions

A smart grid utility company collects electricity consumption readings from 1 million1\text{ million} smart meters every 15 minutes15\text{ minutes}. The total data volume during peak hours reaches 15 MB/s15\text{ MB/s} of XML data. The company needs to ingest this data, convert the format from XML to JSON in near-real-time, and store the output in Amazon S3 for downstream analytics. The ingestion pipeline must scale automatically to handle load fluctuations and require minimal operational overhead. Which combination of actions should the solutions architect recommend to meet these requirements? (Select two.)

  1. Use Amazon Kinesis Data Firehose to ingest the data and invoke an AWS Lambda function to transform the format from XML to JSON.Answer
  2. B
    Ingest the data into an Amazon Kinesis Data Streams stream configured with provisioned mode and a fixed number of shards to handle peak traffic.
  3. Deliver the transformed JSON records from Amazon Kinesis Data Firehose directly to the target Amazon S3 bucket.Answer
  4. D
    Send the readings to an Amazon SQS standard queue to buffer the data and guarantee that messages are processed in strict first-in, first-out (FIFO) order.
  5. E
    Ingest the readings using an Amazon SQS standard queue and trigger an AWS Lambda function to transform the messages while relying on SQS to preserve the exact order of the data.

Answer

The correct combination of actions is to use Amazon Kinesis Data Firehose to ingest the data and invoke an AWS Lambda function to transform the format from XML to JSON, and to deliver the transformed JSON records from Amazon Kinesis Data Firehose directly to the target Amazon S3 bucket.
Amazon Kinesis Data Firehose is a fully managed service that scales automatically to match stream throughput, satisfying the high performance and low operational overhead criteria. It features built-in integration with AWS Lambda to execute data transformation synchronously as data passes through the delivery stream. Delivering the results directly to Amazon S3 completes a robust, scalable, and serverless ingestion pipeline.

Step-by-Step Solution

1
Identify the scaling and operational requirements.
The pipeline must handle up to 15 MB/s15\text{ MB/s} peak traffic, scale automatically, and require minimal operational overhead.
This points toward a serverless managed ingestion service rather than provisioned streams or custom self-managed consumers.
2
Evaluate the ingestion and transformation mechanism.
Amazon Kinesis Data Firehose can ingest the data stream and trigger an inline AWS Lambda function to handle format transformation (XML to JSON) before delivery.
Firehose handles buffering, scaling, and transformation invocation serverlessly, eliminating the need to write custom scaling logic.
3
Select the destination delivery target.
Configure Amazon S3 as the target delivery destination within the Kinesis Data Firehose stream.
Firehose directly writes the output JSON files to S3, completing the end-to-end data flow with zero additional components.

Key Concept

Serverless streaming ingestion and transformation using Amazon Kinesis Data Firehose and AWS Lambda.
Rate this question

Topics

High-Performing Data Ingestion and Transformation Solutions

More questions from this topic