Question

Difficulty: HardImplementing Auto Scaling and Fault Tolerance

An enterprise telemetry analytics SaaS provider runs its data ingestion service on Amazon EC2 instances in an Auto Scaling group (ASG) behind an Application Load Balancer (ALB). The instances process high-velocity telemetry logs from client sites across three Availability Zones. The instances utilize a bootstrap script that downloads configuration files and initializes local caching, which takes approximately 8 minutes to complete before they can accept traffic. The provider recently suffered an outage during an unannounced marketing campaign by one of its enterprise clients, which caused an immediate, massive surge in HTTP traffic. During this surge, the ALB returned 503 Service Unavailable errors. The ASG scale-out policy triggered multiple times, but launched dozens of unnecessary instances that remained idle once the traffic surge subsided. Additionally, outbound database connection attempts from instances in two of the Availability Zones failed because the route tables pointed to a single NAT Gateway located in the third Availability Zone, which became resource-constrained and crashed. Which configuration should a solutions architect implement to resolve these issues?

  1. Establish a NAT Gateway within each Availability Zone and configure the route tables for each private subnet to route outbound traffic through its local gateway. Update the Auto Scaling group to use a target tracking scaling policy with the instance warmup time set to 540 seconds. Coordinate with clients to obtain advance notice of promotional events and request AWS Support to pre-warm the Application Load Balancer.Answer
  2. B
    Route outbound traffic from all subnets through a single NAT Gateway residing in a centralized shared services VPC to control costs. Configure the Auto Scaling group to use simple scaling policies with a cooldown period of 300 seconds. Request AWS Support to pre-warm the Application Load Balancer before expected traffic surges.
  3. C
    Establish a NAT Gateway within each Availability Zone and configure the route tables for each private subnet to route outbound traffic through its local gateway. Update the Auto Scaling group to use a target tracking scaling policy with the instance warmup time set to 300 seconds. Rely on the default automatic scaling capacity of the Application Load Balancer to absorb the sudden traffic surges.
  4. D
    Deploy a single NAT Gateway in one Availability Zone and implement a Gateway Load Balancer to route outbound traffic through a cluster of security appliances. Configure the Auto Scaling group to use a target tracking scaling policy with the instance warmup time set to 540 seconds. Rely on the default automatic scaling capacity of the Application Load Balancer to absorb the sudden traffic surges.

Answer

Establish a NAT Gateway within each Availability Zone and configure the route tables for each private subnet to route outbound traffic through its local gateway. Update the Auto Scaling group to use a target tracking scaling policy with the instance warmup time set to 540 seconds. Coordinate with clients to obtain advance notice of promotional events and request AWS Support to pre-warm the Application Load Balancer.
The correct solution addresses all three failure points. First, setting up a NAT Gateway in each Availability Zone ensures that a failure or capacity limit in one zone does not impact outbound database connections in other zones. Second, configuring the instance warmup time to 540 seconds (which is greater than the 8-minute boot/initialization time) prevents the Auto Scaling group from launching excess instances before the newly scaled-out instances can begin processing traffic and lowering the target metric. Third, obtaining advance warning and pre-warming the Application Load Balancer ensures that the load balancer has enough pre-provisioned capacity to handle the immediate, massive traffic surge without dropping requests or returning 503 errors.

Step-by-Step Solution

1
Address outbound connectivity fault tolerance by provisioning a NAT Gateway in each of the three Availability Zones.
Outbound traffic from each Availability Zone goes through its local NAT Gateway, eliminating cross-AZ dependencies and avoiding a single point of failure.
The original architecture relied on a single NAT Gateway, which crashed due to high traffic and took down outbound connectivity for two other Availability Zones.
2
Modify the Auto Scaling group policy to prevent over-provisioning during instance bootstrapping by adjusting the instance warmup time.
Setting the instance warmup time to 540 seconds (9 minutes) ensures that the group waits for the 8-minute bootstrap process to finish before evaluating scaling metrics again.
If the warmup or cooldown period is shorter than the time required for a new instance to boot and begin processing traffic (8 minutes), the Auto Scaling group will falsely detect that more capacity is needed and launch redundant instances.
3
Establish a process for advance notice of flash traffic spikes and configure load balancer pre-warming.
AWS Support pre-warms the Application Load Balancer to support the expected traffic volume before the spike occurs, preventing 503 Service Unavailable errors.
Application Load Balancers automatically scale in response to traffic, but sudden, massive spikes can overwhelm the load balancer before it can scale up to meet the demand.

Key Concept

To ensure high availability and proper scaling behavior, outbound traffic paths must be redundant across Availability Zones, the Auto Scaling cooldown or warmup times must exceed the initialization time of new instances, and load creators must pre-warm Application Load Balancers for immediate flash traffic.
Rate this question