Soru

Zorluk: Çok zorDisaster Recovery and Business Continuity Execution

An enterprise web application deployed across primary region `us-central1` and secondary recovery region `us-east4` experiences a total regional failure in `us-central1`. The application uses a Pilot Light DR pattern comprising a Cloud SQL PostgreSQL cross-region read replica and a minimal regional Managed Instance Group (MIG) in `us-east4`. To achieve recovery without split-brain data corruption or premature exposure of unvalidated endpoints, in what exact sequence should the SRE team execute the disaster recovery failover runbook tasks?

  1. 1Revoke IAM database write permissions and isolate compute workloads in `us-central1` to prevent dual-writing.
  2. 2Promote the Cloud SQL cross-region read replica in `us-east4` to a standalone writeable primary instance.
  3. 3Scale up the pre-provisioned Compute Engine Managed Instance Group in `us-east4` to target production capacity.
  4. 4Execute automated synthetic transactions and database health checks against the internal load balancer endpoint in `us-east4`.
  5. 5Update the Global External Application Load Balancer backend service to route client ingress traffic to the `us-east4` MIG.

Cevap

The correct operational sequence for regional DR failover is: 1) Isolate `us-central1` compute and database write paths to eliminate split-brain risk, 2) Promote the `us-east4` Cloud SQL cross-region read replica to standalone primary, 3) Scale up the `us-east4` Managed Instance Group to full capacity, 4) Execute synthetic health and data integrity checks against the internal `us-east4` endpoint, and 5) Update Global Load Balancer backends to direct ingress traffic to `us-east4`.
In GCP enterprise DR execution, ensuring data consistency and split-brain prevention is the highest priority. The primary degraded region must first be fenced off by revoking write access. Next, the secondary database replica is promoted to primary so that compute nodes have a valid write target. Then, the pilot-light MIG is scaled to full capacity to accommodate production traffic. Next, internal synthetic validation tests verify system readiness. Finally, the Global Load Balancer backend configuration is updated to route public client traffic to the secondary region.

Adım Adım Çözüm

1
Isolate the primary region
Primary database and compute components are locked from receiving new writes, preventing data divergence.
Before promoting a secondary database, the primary region must be fenced off so that transient network recoveries do not cause split-brain data writes.
2
Promote the database replica
The Cloud SQL replica in `us-east4` becomes an independent primary database.
Application services cannot process state-changing user transactions until a writeable primary database is established.
3
Scale up compute capacity
The pilot-light MIG expands instances to handle production request volumes.
Compute resources must be provisioned and connected to the newly promoted database before receiving production workload demands.
4
Perform synthetic validation
Smoke tests confirm schema integrity, connection pool behavior, and application startup cleanliness.
Directing live user traffic to an unvalidated secondary stack risks mass request drops or silent application failures.
5
Shift global ingress traffic
The Global External Load Balancer routes user requests to the fully healthy secondary region.
Traffic shifting completes the failover procedure once end-to-end operational readiness is validated.

Anahtar Kavram

Disaster Recovery Failover Execution & Split-Brain Prevention
Bu soruyu puanla