Examkin

Question

Difficulty: MediumIncident Management and Automated Alerting

A Site Reliability Engineering (SRE) team is implementing an end-to-end incident management and automated alerting workflow on Google Cloud to handle service level objective (SLO) breaches. In what sequence should the SRE team structure the automated detection, notification, triage, and human escalation pipeline? Arrange the steps in the correct chronological order.

  1. 1Define a Service Level Objective (SLO) in Cloud Monitoring and configure a multi-window burn-rate alert policy.
  2. 2Attach a Cloud Pub/Sub topic as a notification channel to the Cloud Monitoring alert policy.
  3. 3Deploy a Cloud Run event subscriber to process incoming Pub/Sub messages, perform automated diagnostic checks, and apply alert muting rules.
  4. 4Escalate incident details via a webhook integration to an external paging service for primary on-call SRE intervention.

Answer

The correct sequence begins with defining the SLO and burn-rate alert policy in Cloud Monitoring, followed by attaching a Cloud Pub/Sub notification channel, executing automated diagnostic checks and alert muting via a Cloud Run subscriber, and finally escalating to the human on-call responder if manual intervention is required.
The correct workflow adheres to Google SRE best practices for incident response: establishing measurement baselines (SLO burn-rate alerting), publishing incident events asynchronously (Cloud Pub/Sub), executing automated triage and alert suppression (Cloud Run subscriber), and escalating remaining critical incidents to human responders.

Step-by-Step Solution

1
Establish detection baselines
Cloud Monitoring multi-window burn-rate alert policy is configured.
SLO error budget depletion must be monitored before any downstream alerting or event processing can occur.
2
Configure event notification channels
Alert notifications are routed asynchronously to Cloud Pub/Sub.
Pub/Sub decouples metric detection from incident response actions, allowing multiple downstream subscribers to consume alert events.
3
Execute automated triage and alert suppression
Cloud Run subscriber ingests the incident payload, gathers logs, and suppresses noise.
Automated first-response scripts prevent alert storms and enrich incident context prior to paging human operators.
4
Escalate to human responders
On-call engineers receive enriched incident details on their paging system.
Human intervention should be reserved for unresolved or high-impact incidents requiring manual decision-making.

Key Concept

Automated Incident Management and Alert Escalation Lifecycle
Rate this question

Topics

Incident Management and Automated Alerting

More questions from this topic