Examkin

Question

Difficulty: HardServerless and Automated Scaling Architectures for Cost Efficiency

A financial services company is designing a serverless batch data-processing pipeline. External clients upload transaction logs to an Amazon S3 bucket at unpredictable times. The ingestion rate varies from zero uploads during weekends to thousands of simultaneous uploads during end-of-month processing. The pipeline must write metadata to Amazon DynamoDB and execute an analysis script on each uploaded log. The analysis script takes between 2020 and 3030 minutes to complete per log. For audit compliance, transaction logs must be processed in the exact order they are uploaded. Which architecture meets these requirements in the most cost-effective manner?

  1. A
    Ingest uploads via Amazon S3 Event Notifications to a standard Amazon SQS queue. Configure an AWS Lambda function to poll the queue, write metadata to Amazon DynamoDB configured in provisioned capacity mode, and run the analysis script directly within the Lambda function.
  2. B
    Configure Amazon S3 Event Notifications to trigger an AWS Lambda function that writes metadata to Amazon DynamoDB in on-demand capacity mode and sends the message to a standard Amazon SQS queue. Configure an Amazon ECS service running continuously on AWS Fargate to poll the standard SQS queue and run the analysis script.
  3. Configure Amazon S3 Event Notifications to send events to an Amazon SQS FIFO queue. Configure an AWS Lambda function to poll the queue, write metadata to Amazon DynamoDB in on-demand capacity mode, and launch Amazon ECS tasks on AWS Fargate using the Fargate Spot capacity provider to run the analysis script.Answer
  4. D
    Configure Amazon S3 Event Notifications to publish events to an Amazon SNS FIFO topic subscribed to an Amazon SQS FIFO queue. Configure an AWS Lambda function to poll the queue, write metadata to Amazon DynamoDB in provisioned capacity mode, and launch Amazon ECS tasks on AWS Fargate using the Fargate On-Demand capacity provider to run the analysis script.

Answer

Configure Amazon S3 Event Notifications to send events to an Amazon SQS FIFO queue, use AWS Lambda to write metadata to Amazon DynamoDB in on-demand capacity mode, and launch Amazon ECS tasks on AWS Fargate Spot to execute the analysis script.
The correct solution satisfies the compliance requirement by routing event notifications through an Amazon SQS FIFO queue, which guarantees strict ordering. It handles the 2020-to-3030 minute runtime constraint by using AWS Lambda to launch Amazon ECS tasks on AWS Fargate on-demand, bypassing the 1515-minute Lambda execution limit. It optimizes costs by selecting Fargate Spot (which reduces compute costs by up to 70%70\%) and Amazon DynamoDB in on-demand capacity mode (which automatically scales to zero, eliminating idle charges during weekends).

Step-by-Step Solution

1
Select the correct compute service based on workload duration and limits.
Choose Amazon ECS on AWS Fargate instead of running the script directly inside AWS Lambda.
The analysis script takes 2020 to 3030 minutes to complete. AWS Lambda has a strict maximum timeout limit of 1515 minutes, which makes it unsuitable for long-running scripts.
2
Determine the optimal queue configuration for the ordering requirement.
Select Amazon SQS FIFO queues.
The company's audit compliance requires transaction logs to be processed in the exact order they are uploaded. Standard SQS queues only guarantee best-effort ordering, whereas SQS FIFO guarantees first-in, first-out sequence.
3
Choose cost-optimized compute capacity providers and database capacity modes for unpredictable scaling.
Select Fargate Spot capacity providers for ECS tasks, and configure Amazon DynamoDB in on-demand capacity mode.
Fargate Spot offers significant discounts of up to 70%70\% for batch processing that can tolerate interruptions. DynamoDB on-demand mode scales instantly to handle spikes and incurs zero costs when there are no read or write requests during weekends.

Key Concept

Serverless and Automated Scaling Architectures for Cost Efficiency
Rate this question

Topics

Serverless and Automated Scaling Architectures for Cost Efficiency