Examkin

Question

Difficulty: Very hardServerless and Automated Scaling Architectures for Cost Efficiency

A financial services company is redesigning a legacy batch-processing application to reduce costs and automate scaling. The application processes large transaction files uploaded to an Amazon S3 bucket. Each file contains sequential transaction records that must be processed in the exact order of their arrival to prevent account balance discrepancies. The processing logic is compute-intensive, taking between 20 to 45 minutes per file, and the workload is highly unpredictable, with hours of inactivity followed by sudden bursts of files. Which architecture is the most cost-effective and meets the requirements?

  1. A
    Configure Amazon S3 to send event notifications that directly trigger an AWS Lambda function for each uploaded file, adjusting the Lambda memory and ephemeral storage configurations to handle the compute-intensive load.
  2. B
    Configure Amazon S3 to send event notifications to a standard Amazon SQS queue. Deploy the processing application on Amazon EC2 Spot Instances in an Auto Scaling group, scaling the group size based on CPU utilization to minimize compute costs.
  3. Configure Amazon S3 to send event notifications to an Amazon SQS FIFO queue. Deploy the processing application as an Amazon ECS service using AWS Fargate, and configure an AWS Application Auto Scaling policy based on the ApproximateNumberOfMessagesVisible metric to scale Fargate tasks from zero to match the queue depth.Answer
  4. D
    Configure Amazon S3 to send event notifications to an Amazon SQS FIFO queue. Deploy the processing application as an Amazon ECS service on AWS Fargate in private subnets, routing all Amazon S3 file transfers through a NAT Gateway to ensure secure access while scaling Fargate tasks based on queue depth.

Answer

The most cost-effective and compliant architecture is to send S3 event notifications to an Amazon SQS FIFO queue, process the files using Amazon ECS tasks on AWS Fargate, and auto-scale the Fargate tasks from zero based on the queue depth metric.
The correct architecture combines Amazon SQS FIFO queues to satisfy the strict ordering requirements with Amazon ECS on AWS Fargate to handle execution times that exceed the 15-minute AWS Lambda timeout limit. Using AWS Application Auto Scaling to adjust tasks based on queue depth ensures that compute resources scale down to zero during idle periods, maximizing cost-efficiency.

Step-by-Step Solution

1
Evaluate the runtime limits of the compute options.
Identify that the file processing takes 20 to 45 minutes, which exceeds the 15-minute execution limit of AWS Lambda, requiring a containerized compute solution such as Amazon ECS on AWS Fargate.
This rules out direct AWS Lambda invocation as a viable option due to hard timeouts.
2
Analyze the message ordering requirements.
Identify that transaction records must be processed in the exact sequence they arrive to prevent data inconsistency.
This necessitates the use of an Amazon SQS FIFO queue rather than a standard SQS queue, which only offers best-effort ordering.
3
Select the scaling mechanism to optimize costs during idle periods.
Configure AWS Application Auto Scaling to adjust the number of ECS Fargate tasks based on the ApproximateNumberOfMessagesVisible metric of the SQS FIFO queue.
This allows the architecture to scale down to zero tasks when the queue is empty, eliminating idle compute costs during inactive periods.
4
Identify network cost optimization strategies.
Ensure that large data transfers to Amazon S3 do not route through a NAT Gateway, which charges data processing fees.
Routing traffic through a Gateway VPC Endpoint for S3 keeps S3 data transfers free of network charges, whereas a NAT Gateway introduces significant unnecessary costs.

Key Concept

Serverless and Automated Scaling Architectures for Cost Efficiency
Estimated Time:3m 0s
Rate this question

Topics

Serverless and Automated Scaling Architectures for Cost Efficiency