Your organization operates a real-time transaction processing API on Google Cloud. To maintain strict reliability standards, you are implementing an automated incident management and escalation workflow using Google Cloud Observability tools. Order the operational steps sequentially from initial anomaly detection to full incident resolution and notification.
- 1Cloud Monitoring evaluates incoming latency metrics and detects a 14-day SLO error budget burn rate exceeding the critical alerting threshold.
- 2Cloud Monitoring publishes an incident notification event to a designated Pub/Sub topic configured as an automated notification channel.
- 3An event-driven Cloud Run service receives the Pub/Sub event and automatically executes remediation scripts to provision fallback capacity.
- 4Cloud Monitoring observes that latency SLIs fall back below the alert threshold, automatically closes the incident, and notifies the operations team of resolution.
Cevap
The correct sequence starts with Cloud Monitoring detecting an SLO error budget burn rate breach, followed by publishing an alert event to a Pub/Sub topic notification channel, triggering an automated Cloud Run remediation service to restore performance capacity, and concluding with Cloud Monitoring verifying metric recovery and resolving the incident.
In GCP Site Reliability Engineering practices, incident management starts with metric-driven anomaly detection (SLO burn rate evaluation). Upon detecting a breach, Cloud Monitoring uses Pub/Sub notification channels to asynchronously trigger automated remediation code (such as Cloud Run or Cloud Functions). Once the automated action restores service metrics, Cloud Monitoring validates recovery and marks the incident resolved.
Adım Adım Çözüm
Anahtar Kavram
Event-driven automated incident management and escalation pipelines using Cloud Monitoring, Pub/Sub, and Cloud Run.