A smart home security company is designing a high-performing ingestion pipeline for securi… — AWS Certified Solutions Architect - Associate

A smart home security company is designing a high-performing ingestion pipeline for security camera event logs. The pipeline must ingest log payloads from $100,000$ cameras, with each camera sending a $1 \text{ KB}$ JSON payload every $5 \text{ seconds}$ . The solution must process these logs with sub-second latency to perform anomaly detection and trigger immediate alerts. Additionally, all raw logs must be archived in Amazon S3 for historical analysis, partitioned by date. Which two actions should a solutions architect recommend to meet these requirements with the lowest operational overhead? (Select two.)

Configure an Amazon Kinesis Data Stream in On-Demand capacity mode to ingest the camera event logs, and use an AWS Lambda function to process the stream for real-time anomaly detection.Answer
Configure an Amazon Kinesis Data Firehose delivery stream that uses the Kinesis Data Stream as the source to batch and write the raw logs to Amazon S3, using custom prefixes for date-based partitioning.Answer
C
Configure an Amazon Kinesis Data Stream in Provisioned capacity mode with 5 shards to ingest the logs, and consume the stream using an AWS Lambda function for real-time alerting.
D
Configure an Amazon SQS standard queue to ingest the logs from the cameras, and deploy a consumer application on Amazon EC2 instances to process the logs in strict sequential order before archiving them to Amazon S3.
E
Configure Amazon Simple Notification Service (Amazon SNS) to ingest the logs directly from the cameras, and subscribe an Amazon Kinesis Data Firehose delivery stream to route logs to both Amazon S3 and Amazon DynamoDB for real-time queries.

Answer

Configure an Amazon Kinesis Data Stream in On-Demand capacity mode to ingest the camera event logs, and use an AWS Lambda function to process the stream for real-time anomaly detection. Additionally, configure an Amazon Kinesis Data Firehose delivery stream that uses the Kinesis Data Stream as the source to batch and write the raw logs to Amazon S3, using custom prefixes for date-based partitioning.

The correct architecture leverages Amazon Kinesis Data Streams in On-Demand capacity mode to ingest the

20 \text{ MB/s}

stream of security camera logs, which automatically scales to support

20,000

records per second without administrative intervention. An AWS Lambda function consumes this stream to trigger anomalies with sub-second latency. To satisfy the archiving requirement, Amazon Kinesis Data Firehose reads from the same Kinesis Data Stream, batches the records, and writes them to Amazon S3 using date-based partitioning prefixes, optimizing both cost and operational complexity.

Step-by-Step Solution

Calculate the ingestion throughput requirements.

The total ingestion rate is

100,000 \text{ cameras} / 5 \text{ seconds} = 20,000 \text{ records/second}

. The total bandwidth is

20,000 \text{ records/second} \times 1 \text{ KB/record} = 20 \text{ MB/second}

Determining the throughput ensures the chosen ingestion technology is scaled correctly.

Select the ingestion and real-time processing technology.

Amazon Kinesis Data Streams in On-Demand capacity mode automatically handles up to

200 \text{ MB/s}

and

200,000

records/second of write capacity. An AWS Lambda function is configured to read from this stream to achieve sub-second latency alerts.

On-Demand mode prevents ProvisionedThroughputExceededException and minimizes management overhead.

Select the historical archiving mechanism.

An Amazon Kinesis Data Firehose delivery stream is configured to consume the data from the Kinesis Data Stream and write it to Amazon S3 in batches.

Firehose handles partitioning and batching automatically with minimal operational effort.

Key Concept

Decoupling high-performance stream ingestion for real-time processing and managed archiving using Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose.

Question

Answer

Step-by-Step Solution

Key Concept

Common Mistakes

Topics

More questions from this topic