Soru

Zorluk: OrtaIncident Management and Automated Alerting

An e-commerce platform processes checkout transactions through a payment gateway microservice hosted on Google Cloud. During brief, transient database latency spikes, the Site Reliability Engineering (SRE) team receives excessive false-positive incident alerts, while genuine sustained error budget depletion is occasionally detected too late. The team needs to redesign their Cloud Monitoring automated alerting strategy to reliably detect severe outages quickly while preventing alert fatigue from transient spikes. Which architecture approach should be implemented?

  1. Configure Cloud Monitoring alerting policies using multi-window, multi-threshold burn rate monitoring based on the service SLO.Cevap
  2. B
    Set static metric threshold alerts on raw 500-series HTTP response counts with a 1-minute aggregation window.
  3. C
    Configure the load balancer health check probes to execute full end-to-end payment database transaction queries.
  4. D
    Assign the primitive Owner role to the automated Incident Manager service account to allow unrestricted incident remediation actions.

Cevap

Configure Cloud Monitoring alerting policies using multi-window, multi-threshold burn rate monitoring based on the service SLO.
Configuring multi-window, multi-threshold burn rate alerting in Cloud Monitoring is the Google Cloud recommended best practice for SRE incident management. It evaluates error budget consumption across multiple time windows (such as a 5-minute window for high burn rates and a 1-hour window for sustained lower burn rates), ensuring rapid notification for true incidents while filtering out noise from brief, transient downstream spikes.

Adım Adım Çözüm

1
Analyze alerting requirements and operational goals
Identified the need to balance fast detection of catastrophic outages with prevention of false positives from short, transient spikes.
Traditional static threshold alerting suffers from high noise or delayed response depending on how the threshold is set.
2
Evaluate Google Cloud SRE alerting best practices
Selected multi-window, multi-threshold burn rate alerting on Service Level Indicators (SLIs) and Service Level Objectives (SLOs).
Multi-window burn rate alerts trigger when error budget consumption rates exceed defined ratios over both brief (e.g., 5-minute) and sustained (e.g., 1-hour) time frames.
3
Validate security and architecture constraints
Confirmed that multi-window burn rate alerting integrates natively with Cloud Monitoring without exposing backend databases to health check cascades or requiring excessive IAM privileges.
Avoids anti-patterns such as deep dependency health check probes and primitive IAM role assignments.

Anahtar Kavram

Multi-window multi-threshold SLO burn rate alerting in Incident Management
Bu soruyu puanla