A financial services company is designing a serverless batch data-processing pipeline. External clients upload transaction logs to an Amazon S3 bucket at unpredictable times. The ingestion rate varies from zero uploads during weekends to thousands of simultaneous uploads during end-of-month processing. The pipeline must write metadata to Amazon DynamoDB and execute an analysis script on each uploaded log. The analysis script takes between and minutes to complete per log. For audit compliance, transaction logs must be processed in the exact order they are uploaded. Which architecture meets these requirements in the most cost-effective manner?
- AIngest uploads via Amazon S3 Event Notifications to a standard Amazon SQS queue. Configure an AWS Lambda function to poll the queue, write metadata to Amazon DynamoDB configured in provisioned capacity mode, and run the analysis script directly within the Lambda function.
- BConfigure Amazon S3 Event Notifications to trigger an AWS Lambda function that writes metadata to Amazon DynamoDB in on-demand capacity mode and sends the message to a standard Amazon SQS queue. Configure an Amazon ECS service running continuously on AWS Fargate to poll the standard SQS queue and run the analysis script.
- Configure Amazon S3 Event Notifications to send events to an Amazon SQS FIFO queue. Configure an AWS Lambda function to poll the queue, write metadata to Amazon DynamoDB in on-demand capacity mode, and launch Amazon ECS tasks on AWS Fargate using the Fargate Spot capacity provider to run the analysis script.Cevap
- DConfigure Amazon S3 Event Notifications to publish events to an Amazon SNS FIFO topic subscribed to an Amazon SQS FIFO queue. Configure an AWS Lambda function to poll the queue, write metadata to Amazon DynamoDB in provisioned capacity mode, and launch Amazon ECS tasks on AWS Fargate using the Fargate On-Demand capacity provider to run the analysis script.
Cevap
Configure Amazon S3 Event Notifications to send events to an Amazon SQS FIFO queue, use AWS Lambda to write metadata to Amazon DynamoDB in on-demand capacity mode, and launch Amazon ECS tasks on AWS Fargate Spot to execute the analysis script.
The correct solution satisfies the compliance requirement by routing event notifications through an Amazon SQS FIFO queue, which guarantees strict ordering. It handles the -to- minute runtime constraint by using AWS Lambda to launch Amazon ECS tasks on AWS Fargate on-demand, bypassing the -minute Lambda execution limit. It optimizes costs by selecting Fargate Spot (which reduces compute costs by up to ) and Amazon DynamoDB in on-demand capacity mode (which automatically scales to zero, eliminating idle charges during weekends).
Adım Adım Çözüm
Anahtar Kavram
Serverless and Automated Scaling Architectures for Cost Efficiency