Question

Difficulty: HardDefining and Managing SLOs, SLIs, and Error Budgets

A healthcare telemetry platform hosted on Google Cloud Run and Cloud Bigtable processes real-time patient metrics. The Site Reliability Engineering (SRE) team defines an availability Service Level Objective (SLO) of 99.9%99.9\% measured over a rolling 30-day window (43,20043,200 minutes). During an unexpected deployment failure, the service suffered complete downtime for 1818 minutes. Later in the same rolling window, a database connection pool exhaustion caused a partial degradation for 6060 minutes, during which 40%40\% of all incoming telemetry requests failed. Assuming a constant request rate throughout the window, how many minutes of error budget remain for this 30-day period?

Answer: 1.2 minutes

Answer

The remaining error budget for the rolling 30-day window is 1.21.2 minutes.
For a 30-day window (43,20043,200 minutes) with a 99.9%99.9\% SLO target, the total allowable error budget is 0.1%×43,200=43.20.1\% \times 43,200 = 43.2 minutes. The complete outage consumed 1818 minutes. The partial degradation consumed 60×0.40=2460 \times 0.40 = 24 minutes of equivalent downtime budget. Total consumed error budget is 18+24=4218 + 24 = 42 minutes. Therefore, the remaining error budget is 43.242=1.243.2 - 42 = 1.2 minutes.

Step-by-Step Solution

1
Calculate the total allowable error budget in minutes for the 30-day window
Total error budget = 43,200 minutes×(10.999)=43.2 minutes43,200 \text{ minutes} \times (1 - 0.999) = 43.2 \text{ minutes}.
An SLO of 99.9%99.9\% allows for an unreliability margin (error budget) of 0.1%0.1\% (0.0010.001) over the 43,20043,200-minute window.
2
Determine error budget consumed during the full outage
Full outage budget consumed = 18 minutes×100%=18.0 minutes18 \text{ minutes} \times 100\% = 18.0 \text{ minutes}.
During complete downtime (100%100\% request failure rate), every minute counts fully against the error budget.
3
Determine error budget consumed during the partial degradation
Partial outage budget consumed = 60 minutes×40%=24.0 minutes60 \text{ minutes} \times 40\% = 24.0 \text{ minutes}.
Under constant traffic conditions, partial failures consume error budget proportionally to the error rate (60×0.40=24.060 \times 0.40 = 24.0 minutes).
4
Calculate total consumed error budget and remaining budget
Total consumed = 18.0+24.0=42.0 minutes18.0 + 24.0 = 42.0 \text{ minutes}. Remaining error budget = 43.242.0=1.2 minutes43.2 - 42.0 = 1.2 \text{ minutes}.
Subtracting the total consumed downtime equivalent (42.042.0 minutes) from the allowable budget (43.243.2 minutes) yields the remaining error budget.

Key Concept

Error Budget Calculation for Service Level Objectives
Estimated Time:2m 0s
Rate this question