Question

Difficulty: MediumServerless and Automated Scaling Architectures for Cost Efficiency

A gaming company needs to process game session logs uploaded to an Amazon S3 bucket. The processing application runs as a containerized workload, with each log file taking 20 to 30 minutes to process. The logs must be processed in the exact order they are uploaded to maintain chronological player states in Amazon DynamoDB. The volume of uploads is highly unpredictable, with significant spikes during weekends and virtually no activity during weekdays. Which architecture is the most cost-effective and meets these requirements?

  1. Configure Amazon S3 to send event notifications to an Amazon SQS FIFO queue. Run the containerized application on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate, scaling tasks based on the queue size, and store the state in Amazon DynamoDB configured with on-demand capacity mode.Answer
  2. B
    Configure Amazon S3 to send event notifications to a standard Amazon SQS queue. Run the containerized application on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate, scaling tasks based on the queue size, and store the state in Amazon DynamoDB configured with on-demand capacity mode.
  3. C
    Configure Amazon S3 to trigger an AWS Lambda function directly to download and process the log files, and store the state in Amazon DynamoDB configured with on-demand capacity mode.
  4. D
    Configure Amazon S3 to send event notifications to an Amazon SQS FIFO queue. Run the containerized application on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate, scaling tasks based on the queue size, and store the state in Amazon DynamoDB configured with provisioned capacity mode.

Answer

Configure Amazon S3 to send event notifications to an Amazon SQS FIFO queue. Run the containerized application on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate, scaling tasks based on the queue size, and store the state in Amazon DynamoDB configured with on-demand capacity mode.
The correct architecture uses Amazon SQS FIFO to guarantee that log files are processed in chronological order. Since the processing jobs take 20 to 30 minutes, they exceed the 15-minute execution limit of AWS Lambda, requiring containerized execution on AWS Fargate. AWS Fargate scales tasks dynamically based on queue activity to ensure cost efficiency. Finally, using Amazon DynamoDB in on-demand capacity mode is the most cost-effective option for highly unpredictable workloads with extended idle periods, as it charges only for read and write requests actually performed rather than provisioning unused capacity.

Step-by-Step Solution

1
Evaluate the execution duration requirement against compute options.
Since each log processing job takes 20 to 30 minutes, AWS Lambda (which has a 15-minute execution limit) is ruled out. Containerized execution on AWS Fargate is chosen as the compute platform.
AWS Fargate allows containerized tasks to run without time limits while scaling dynamically and avoiding the management of underlying EC2 instances.
2
Determine the decoupling and ordering mechanism.
Amazon SQS FIFO queue is selected to receive S3 event notifications.
The scenario requires logs to be processed in the exact order they are uploaded to maintain chronological state. Only SQS FIFO guarantees first-in, first-out delivery, whereas standard SQS does not.
3
Select the database capacity mode for the highly variable workload.
Configure Amazon DynamoDB with on-demand capacity mode.
Since the workload is highly unpredictable with zero activity on weekdays and high spikes on weekends, on-demand capacity mode is the most cost-effective as it charges only for actual read and write requests and has zero idle compute/capacity cost.

Key Concept

Combining serverless and automated scaling compute (Fargate), ordered messaging (SQS FIFO), and on-demand database capacity (DynamoDB) to optimize costs for unpredictable, long-running batch workloads.
Estimated Time:2m 0s
Rate this question