Question

Difficulty: MediumIncident Management and Automated Alerting

An online media streaming company runs a video metadata microservice on Google Cloud. During peak traffic hours, brief network latency spikes of 5 to 10 seconds periodically occur, triggering temporary error rate spikes that resolve automatically. However, the operations team receives dozens of redundant PagerDuty alerts every night, causing severe alert fatigue. You need to configure a Cloud Monitoring automated alerting policy that ignores these transient spikes while ensuring sustained service degradations immediately trigger incident notifications. Which alerting policy configuration should you implement?

  1. Configure a metric threshold alerting condition with a rolling alignment period of 5 minutes and set the duration condition to require the threshold to be breached continuously for 5 minutes before triggering an alert.Answer
  2. B
    Configure an instantaneous metric threshold alert condition with a duration of 0 minutes to ensure immediate incident creation upon any detected error spike.
  3. C
    Create a Cloud Logging exclusion filter for high-severity application error logs generated during peak traffic hours to suppress log ingestion volume.
  4. D
    Modify the HTTP Load Balancer health check path to execute a synchronous backend database query on every probe request.

Answer

Configure a metric threshold alerting condition with a rolling alignment period and set the duration condition to require the threshold to be breached continuously for several minutes before triggering an alert.
In Google Cloud Monitoring, configuring a metric threshold alert with a rolling alignment period and setting a duration window (such as 5 minutes continuously) ensures that brief metric spikes are filtered out. An automated incident notification is generated only when the metric stays above the defined threshold continuously for the entire duration, eliminating alert fatigue while reliably detecting real outages.

Step-by-Step Solution

1
Identify the operational challenge.
Observed that transient 5-10 second metric spikes trigger redundant incident notifications, resulting in alert fatigue.
Alerting policies evaluated on raw, instantaneous data points fire immediately without validating sustained impact.
2
Analyze Cloud Monitoring alerting condition parameters.
Determined that using a rolling alignment window and requiring a continuous duration condition prevents transient noise from triggering notifications.
A duration window ensures the monitored metric remains above the threshold for the entire period before firing an alert.
3
Select the appropriate architecture configuration.
Selected the option requiring rolling alignment and continuous metric threshold breach across a multi-minute duration.
This configuration balances noise reduction for transient events with reliable alerting for true service outages.

Key Concept

Cloud Monitoring Alert Policy Duration and Alignment Windows
Rate this question