Question

Difficulty: HardDisaster Recovery and Business Continuity Execution

A global media streaming platform hosts its live session state and metadata processing engine on Google Cloud across two regions. The primary active region is us-west1 and the passive disaster recovery region is us-east1. The architecture utilizes Cloud Bigtable for high-throughput session state, Google Kubernetes Engine (GKE) for stateless microservices, and Cloud DNS routing policies for external traffic management. A catastrophic zone-wide power failure has disabled the us-west1 infrastructure. What is the correct sequence of steps to execute the regional failover runbook to restore full operational capacity in us-east1 while maintaining data consistency?

  1. 1Update Cloud DNS routing policies to temporarily drain incoming user requests and prevent dirty writes to the degraded us-west1 region.
  2. 2Reconfigure the Cloud Bigtable application profile to use single-cluster routing targeting the us-east1 cluster with single-row consistency enabled.
  3. 3Scale up the GKE node pool and workload replicas in us-east1 utilizing pre-reserved capacity to accommodate the full production workload volume.
  4. 4Validate backend Service Level Indicators (SLIs) in Cloud Monitoring and direct live production traffic to the us-east1 External HTTP(S) Load Balancer.

Answer

The correct operational sequence begins by updating Cloud DNS to isolate ingress traffic from us-west1, followed by reconfiguring the Cloud Bigtable application profile to single-cluster routing in us-east1, then scaling GKE compute resources in us-east1 using capacity reservations, and finally validating SLIs before routing live production traffic to us-east1.
Executing a disaster recovery failover requires strict adherence to dependency order: first, isolate traffic to halt invalid writes to the primary region; second, reconfigure stateful database application profiles to target the standby region; third, scale compute resources using capacity reservations; fourth, verify monitoring health metrics before opening live traffic ingress to the secondary region.

Step-by-Step Solution

1
Isolate the failing region by updating Cloud DNS routing policy.
Prevents ongoing transactions from failing or causing data inconsistency in us-west1.
Traffic isolation must precede failover actions to stop split-brain state creation.
2
Reconfigure Cloud Bigtable application profile for single-cluster routing to us-east1.
Ensures all incoming state updates target the healthy us-east1 Bigtable cluster.
Stateful data tier routing must be explicitly reconfigured before compute microservices attempt writes.
3
Scale the secondary GKE deployment in us-east1 to full production capacity.
Secondary compute nodes and pods expand to process 100% of production traffic.
Stateless compute capacity must be fully provisioned while the database tier is ready.
4
Verify service metrics in Cloud Monitoring and shift external load balancer ingress traffic.
Full production traffic is safely served out of us-east1 without downtime or data corruption.
Validation of health check endpoints and SLIs ensures the failover was successful before public traffic cutover.

Key Concept

Disaster Recovery Execution and Regional Failover Ordering
Rate this question