An enterprise financial audit platform processes transaction verification requests using a GraphQL API on Google Kubernetes Engine (GKE). The Site Reliability Engineering (SRE) team needs to establish an alerting strategy to preserve the service's availability Service Level Objective (SLO) of 99.9% over a 30-day rolling window. The team wants to ensure they receive immediate notifications for rapid error budget depletion while avoiding alert fatigue from minor, transient error spikes. Which alerting configuration should the SRE team implement?
- Implement multi-window, multi-burn-rate alerting that measures short-term and long-term error budget consumption rates.Answer
- BConfigure a static threshold alert on Cloud Monitoring whenever HTTP 5xx error rates exceed 0.1% over a 5-minute window.
- CSet up an alert on GKE node CPU utilization when aggregate node pool usage exceeds 85% for more than 10 minutes.
- DDefine the 99.9% successful response target as the Service Level Indicator (SLI) and alert when total HTTP 5xx errors exceed 100 in an hour.
Answer
The SRE team should implement multi-window, multi-burn-rate alerting that measures short-term and long-term error budget consumption rates.
Implementing multi-window, multi-burn-rate alerting is the recommended GCP SRE best practice for managing error budgets. By monitoring multiple time windows simultaneously (such as short windows for fast burn and long windows for slow burn), SRE teams can quickly catch catastrophic failures before the error budget is exhausted while avoiding noisy alerts from brief, transient error spikes.
Step-by-Step Solution
Key Concept
Multi-window, multi-burn-rate alerting for error budgets
Estimated Time:1m 30s