An Site Reliability Engineering (SRE) team is managing a fleet of GKE microservices that output custom application logs to Cloud Logging. The team needs to configure automated real-time incident routing whenever application logs register a severity of `ERROR`. The solution must trigger an alerting mechanism that sends notifications directly to a Cloud Pub/Sub topic for downstream automated remediation. Which TWO actions must the team perform to achieve this requirement using Google-recommended practices?
- Create a custom log-based metric in Cloud Logging with a log filter defined for severity `ERROR`.Answer
- Configure an alerting policy in Cloud Monitoring that monitors the log-based metric threshold and targets a Pub/Sub notification channel.Answer
- CInstall and configure the legacy Stackdriver Logging Agent on all GKE cluster worker nodes to stream log records directly to Cloud Pub/Sub.
- DGrant the Project Editor (`roles/editor`) primitive role to the Pub/Sub service account to enable event publishing permissions.
Answer
To implement real-time incident routing based on error log entries, the SRE team must create a log-based counter metric in Cloud Logging filtered for `ERROR` severity, and then create a Cloud Monitoring alerting policy evaluating this metric that sends alerts to a Pub/Sub notification channel.
To trigger automated workflows from log events in Google Cloud, log entries must first be converted into metric data using a custom log-based metric in Cloud Logging. Once the metric exists, a Cloud Monitoring alerting policy can monitor the metric stream and publish alert notifications directly to a Cloud Pub/Sub topic channel.
Step-by-Step Solution
Key Concept
Log-based Metrics and Cloud Monitoring Alerting Policies with Pub/Sub Notification Channels