Question

Difficulty: MediumServerless and Automated Scaling Architectures for Cost Efficiency

A biotechnology company runs computational workflows that simulate molecular interactions. The simulations must be processed in the exact chronological order in which they are submitted to ensure data dependency integrity. Each simulation takes between 11 and 22 hours to execute. The simulation requests are highly irregular, with dozens of submissions occurring simultaneously followed by days of complete inactivity. The database must scale instantly to handle the write throughput when simulations complete, but must not incur compute costs during inactive periods. Which architecture is the most cost-effective and meets these requirements?

  1. Queue the jobs in an Amazon SQS FIFO queue, execute the simulations as Amazon ECS tasks on AWS Fargate, and store the output in an Amazon DynamoDB table configured with on-demand capacity.Answer
  2. B
    Queue the jobs in an Amazon SQS standard queue, execute the simulations as AWS Lambda functions, and store the output in an Amazon DynamoDB table configured with provisioned capacity.
  3. C
    Queue the jobs in an Amazon SQS standard queue, execute the simulations as Amazon ECS tasks on AWS Fargate, and store the output in an Amazon DynamoDB table configured with on-demand capacity.
  4. D
    Queue the jobs in an Amazon SQS FIFO queue, execute the simulations as Amazon ECS tasks on AWS Fargate, and store the output in an Amazon DynamoDB table configured with provisioned capacity.

Answer

Queue the jobs in an Amazon SQS FIFO queue, execute the simulations as Amazon ECS tasks on AWS Fargate, and store the output in an Amazon DynamoDB table configured with on-demand capacity.
The correct solution uses an Amazon SQS FIFO queue to enforce strict ordering of the jobs. For the compute layer, Amazon ECS on AWS Fargate is chosen because the simulation runtime of 1 to 2 hours exceeds the 15-minute limitation of AWS Lambda. Fargate is serverless and scales to zero, ensuring zero compute cost when idle. For the database, Amazon DynamoDB in on-demand capacity mode instantly scales to handle completion writes and avoids ongoing idle costs during periods of inactivity.

Step-by-Step Solution

1
Analyze the ordering requirement.
Since the simulation jobs must be processed in the exact chronological order of submission, an Amazon SQS FIFO (First-In-First-Out) queue is required, eliminating standard SQS options.
Amazon SQS standard queues do not guarantee message ordering, whereas SQS FIFO queues ensure strict order-of-arrival processing.
2
Analyze the compute runtime and serverless scaling requirement.
AWS Fargate must be used instead of AWS Lambda.
Each simulation runs for 1 to 2 hours, which exceeds the 15-minute execution limit of AWS Lambda. Running containerized tasks on AWS Fargate allows execution times up to several days while remaining serverless and scaling to zero when idle.
3
Analyze the database capacity mode for the irregular workload.
Amazon DynamoDB must be configured with on-demand capacity mode.
On-demand capacity mode dynamically scales to handle instant write spikes and does not charge for idle capacity during inactive periods, whereas provisioned capacity mode would incur ongoing costs for idle resources.

Key Concept

Selecting cost-effective serverless compute and database capacity modes for irregular, long-running tasks requiring ordered processing.
Rate this question