A financial service provider is designing a real-time fraud detection and auditing solutio… — AWS Certified Solutions Architect - Associate

A financial service provider is designing a real-time fraud detection and auditing solution. The solution must ingest transaction logs from millions of globally distributed POS terminals, with peak traffic reaching $15\text{ MB/s}$ and up to $15,000$ transactions per second. The transactions must be processed in the exact order they are received for each terminal to ensure accurate validation. The raw transactions must also be stored in Apache Parquet format on Amazon S3 with low operational overhead.

Which combination of actions should the solutions architect recommend to meet these requirements? (Select two.)

Create an Amazon Kinesis Data Stream in Provisioned mode with $15$ shards, and write transactions using the terminal ID as the partition key.Cevap
Create an Amazon Data Firehose delivery stream with Amazon S3 as the destination, set the Kinesis Data Stream as the source, and enable data format conversion to Apache Parquet using an AWS Glue table.Cevap
C
Create an Amazon SQS standard queue to ingest the transaction logs from the POS terminals, and configure an AWS Lambda function to process the logs.
D
Create an Amazon Kinesis Data Stream in Provisioned mode with $5$ shards, and rely on Kinesis Auto Scaling to dynamically provision additional shards as throughput increases.
E
Create an Amazon Kinesis Data Stream in Provisioned mode with $15$ shards, and write transactions using the transaction timestamp as the partition key.

Cevap

Create an Amazon Kinesis Data Stream in Provisioned mode with

15

shards using the terminal ID as the partition key, and create an Amazon Data Firehose delivery stream using the Kinesis Data Stream as the source with inline conversion to Apache Parquet using AWS Glue.

To ingest transaction logs at a peak rate of

15\text{ MB/s}

and

15,000

transactions per second while maintaining strict order per terminal, an Amazon Kinesis Data Stream in Provisioned mode with at least

15

shards is required. Using the terminal ID as the partition key ensures that records for each terminal are routed to the same shard in order. For the transformation and S3 delivery, Amazon Data Firehose can ingest from the Kinesis Data Stream and perform inline data format conversion to Apache Parquet using AWS Glue with minimal operational overhead.

Adım Adım Çözüm

Calculate the required ingestion throughput.

With a peak of

15\text{ MB/s}

and

15,000

transactions per second, and knowing that a single Kinesis shard supports

1\text{ MB/s}

1,000

records per second, the stream requires a minimum of

15

shards.

Provisioning fewer shards will cause write throttling and data loss during peak times.

Ensure chronological order of transactions per terminal.

Using the terminal ID as the partition key ensures that all transactions from a specific terminal are sent to the same shard and processed in order.

A monotonically increasing timestamp key or generic keys would distribute the same terminal's transactions across multiple shards or cause hot shards, violating ordering.

Implement low-overhead transformation and delivery.

Configure Amazon Data Firehose with the Kinesis Data Stream as the source and S3 as the destination. Firehose automatically converts the incoming JSON data to Apache Parquet inline using an AWS Glue schema before writing to S3.

This serverless configuration scales dynamically and requires no custom code, minimizing operational overhead.

Anahtar Kavram

High-Performing Data Ingestion and Transformation Solutions

Soru

Cevap

Adım Adım Çözüm

Anahtar Kavram

Sık Yapılan Hatalar

Konular

Bu konudan daha fazla soru