A video streaming platform hosts its user authentication and session management microservice on Google Cloud Run. The Site Reliability Engineering (SRE) team defines a monthly availability Service Level Objective (SLO) of based on the ratio of successful HTTP responses ( and ) to total requests. To prevent alert fatigue while ensuring fast detection of rapid error budget consumption, which alerting strategy should the SRE team implement?
- Configure multi-window, multi-burn-rate alerts in Cloud Monitoring to trigger when short-term (e.g., 5-minute and 1-hour) and long-term (e.g., 6-hour and 3-day) budget consumption rates exceed defined thresholds.Answer
- BSet up a static threshold alert in Cloud Monitoring that triggers whenever the HTTP 5xx error rate exceeds 0.1% over a single rolling 5-minute window.
- CCreate an automated metric alert that triggers whenever Cloud Run CPU utilization exceeds 85% for more than 10 minutes.
- DTrigger a high-priority alert whenever the measured Service Level Indicator (SLI) value falls below the 99.9% target Service Level Objective (SLO) for any single minute.
Answer
Configure multi-window, multi-burn-rate alerts in Cloud Monitoring to trigger when short-term (e.g., 5-minute and 1-hour) and long-term (e.g., 6-hour and 3-day) budget consumption rates exceed defined thresholds.
Configuring multi-window, multi-burn-rate alerts in Cloud Monitoring is the Google SRE standard for monitoring SLO error budget consumption. It evaluates both short-term lookback windows (to quickly detect catastrophic outages) and long-term lookback windows (to detect subtle, persistent bugs draining the budget over days), while drastically reducing alert noise.
Step-by-Step Solution
Key Concept
Multi-window, multi-burn-rate alerting for SLO error budgets