A financial company runs a daily audit process that analyzes transaction logs uploaded to an Amazon S3 bucket. The daily processing workload is highly variable: of the log files are small and take less than minutes to process, while the remaining of the files contain aggregated transaction records and can take up to minutes to process. The audit process must scale dynamically to handle volume spikes during quarter-end reporting and minimize idle compute costs.
Which combination of actions should a solutions architect recommend to meet these requirements most cost-effectively? (Select TWO.)
- Trigger AWS Lambda functions from Amazon S3 Event Notifications to process the files that take less than minutes to run.Answer
- Submit containerized processing jobs to AWS Batch running on AWS Fargate Spot for files that take longer than minutes to run.Answer
- CConfigure AWS Lambda functions with a -minute execution timeout to process all files regardless of size.
- DUse an Amazon SQS standard queue to buffer and ingest events, relying on it to guarantee first-in, first-out (FIFO) ordering of the transaction files during parallel processing.
- EDeploy the compute instances in a private subnet and route all traffic to Amazon S3 through a NAT Gateway to minimize data transfer costs.
Answer
The most cost-effective solution uses AWS Lambda to process the short-running files (under minutes) and AWS Batch on AWS Fargate Spot to process the long-running files (up to minutes).
For the files that take less than minutes to run, AWS Lambda provides a fully managed, serverless execution model that scales automatically with the input and charges only for execution time. For the aggregated transaction files taking up to minutes, AWS Lambda cannot be used due to its hard timeout limit of minutes. Packaging these jobs into containers and running them via AWS Batch using AWS Fargate Spot capacity providers allows the jobs to run without timeout limits while avoiding idle compute costs and achieving up to a discount compared to On-Demand rates.
Step-by-Step Solution
Key Concept
Using a hybrid serverless model to handle variable workloads, aligning compute choice (AWS Lambda vs. AWS Batch on Fargate Spot) with execution time limits to optimize cost and scalability.