Examkin

Question

Difficulty: MediumDisaster Recovery and Business Continuity Execution

A global telematics company operates an automated fleet tracking system in Google Cloud. The primary streaming pipeline runs in `us-central1`, ingesting real-time data via Cloud Pub/Sub, processing it with Cloud Dataflow, and storing state in a regional Cloud Bigtable cluster. In the event of a catastrophic failure in `us-central1`, the operational recovery runbook requires executing a controlled regional failover to `us-east4` with zero data loss for persistent state. What is the correct sequential order of operational steps to execute this disaster recovery failover runbook?

  1. 1Redirect client telemetry traffic to the secondary Cloud Pub/Sub topic endpoint in us-east4 using Cloud DNS routing policy adjustments.
  2. 2Drain the existing Cloud Dataflow streaming pipeline in us-central1 to ensure all inflight telemetry records are fully processed into Cloud Bigtable.
  3. 3Promote the secondary Cloud Bigtable cluster in us-east4 to serve application traffic by updating the Bigtable app profile routing rules.
  4. 4Launch the failover Cloud Dataflow streaming job in us-east4 pointing to the secondary Pub/Sub subscription and the us-east4 Bigtable cluster.

Answer

The correct operational failover sequence begins by redirecting client ingress traffic via Cloud DNS to the secondary Cloud Pub/Sub topic, followed by draining the primary Cloud Dataflow pipeline to preserve inflight data, promoting the target Cloud Bigtable cluster app profile to us-east4, and finally launching the failover Cloud Dataflow processing job in us-east4.
Executing regional failover for streaming systems requires a strict sequence: first, redirect client traffic at the ingress layer (DNS/PubSub) so new data buffers safely in the secondary region; second, drain the degraded region's stream processing engine (Dataflow) to preserve inflight data and adhere to RPO zero targets; third, promote the replicated database layer (Cloud Bigtable) by updating routing app profiles; fourth, spin up the secondary stream processing workers in the recovery region to resume normal operations.

Step-by-Step Solution

1
Adjust Cloud DNS traffic routing to point client devices to the secondary ingestion Pub/Sub topic in us-east4.
New telemetry stream data is safely buffered in the failover region's Pub/Sub topic.
Immediate DNS traffic redirection stops new workload accumulation in the failing region and prevents client connection timeouts.
2
Issue a drain command to the primary Cloud Dataflow streaming pipeline operating in us-central1.
Inflight records stored in Pub/Sub and Dataflow pipeline buffers are processed and committed to Cloud Bigtable before job termination.
Draining ensures RPO requirements are met without dropping records currently residing in pipeline memory.
3
Update Cloud Bigtable App Profiles to route application traffic to the replicated instance cluster in us-east4.
The database storage tier in us-east4 is configured as the active target for read/write queries.
Storage cluster app profiles must be switched after inflight primary writes are finished to prevent split-brain conditions or data inconsistency.
4
Start the secondary Cloud Dataflow streaming job in us-east4 reading from the failover Pub/Sub subscription.
Stream ingestion and processing fully resume in the failover region.
The failover data processing pipeline can safely start once both the ingress queue and storage layer are properly configured in the secondary region.

Key Concept

Disaster Recovery Execution Order for Streaming and Storage Workloads
Rate this question

Topics

Disaster Recovery and Business Continuity Execution

More questions from this topic