An enterprise healthcare organization hosts a mission-critical API gateway on Google Cloud for processing patient records. The operations team needs to establish an automated incident management pipeline that alerts on-call engineers and creates incident tickets during genuine operational disruptions. The system must notify teams only when sustained error rates consume the monthly reliability budget, while ignoring brief 1-minute network glitches to prevent alert fatigue. Which strategy should the team implement in Google Cloud Monitoring?
- AConfigure a static metric threshold alerting policy based on 1-minute HTTP 5xx log counts that sends HTTP webhook calls directly to the ticketing platform whenever any error occurs.
- Create an alerting policy using multi-window error budget burn-rate conditions tied to the service SLO, routed via Cloud Pub/Sub to trigger an automated ticket creation service.Answer
- CConfigure an Application Load Balancer health check that executes deep database queries every 5 seconds, and trigger incident tickets whenever backend targets fail health probes.
- DAssign the primitive Project Owner role to a service account so Cloud Monitoring can execute automated remediation scripts and generate tickets with unrestricted permissions across all resources.
Answer
Create an alerting policy using multi-window error budget burn-rate conditions tied to the service SLO, routed via Cloud Pub/Sub to trigger an automated ticket creation service.
The correct approach uses multi-window error budget burn-rate alerting in Google Cloud Monitoring. Burn-rate alerts evaluate both short-term and long-term consumption rates of the Service Level Objective (SLO) error budget. Routing alerts through Cloud Pub/Sub ensures reliable, asynchronous execution of downstream ticket creation without exposing services to tight coupling or alert floods.
Step-by-Step Solution
Key Concept
Multi-window error budget burn-rate alerting for SRE incident management