An organization manages a fleet of 500 Compute Engine virtual machines hosting a latency-sensitive trading platform across multiple Google Cloud projects. The infrastructure engineering team implemented a unified log exclusion filter at the organization level (`resource.type="gce_instance" AND severity<ERROR`) on an aggregate Log Router sink to reduce Cloud Logging storage ingestion overhead. Shortly after applying this filter, automated incident response alerts stopped triggering for VM crash loops and system failures caused by memory exhaustion, even though the application services were uncommunicative. Upon investigation, engineers noticed that critical Linux kernel panic logs and systemd process failure events emitted by the OS were missing from their Monitoring dashboards and BigQuery security audit datasets. Which architectural modification should the team implement to restore operational visibility and automated incident alerting while retaining log storage cost control?
- ARemove the organization-level exclusion filter and configure non-billable Log Router exclusion rules specifically at the project bucket level using log analytics sinks while routing system metrics via Cloud Monitoring.
- Modify the organization log exclusion filter to explicitly exclude log messages only matching `severity<WARNING AND NOT logName:"logs/systemd" AND NOT logName:"logs/syslog"`, ensuring high-priority system daemon events bypass exclusion filters.Answer
- CGrant `roles/logging.admin` to the Compute Engine default service account across all instances and deploy the Ops Agent configured with default syslog stream settings.
- DCreate an additional organization-level sink pointing directly to a Cloud Pub/Sub topic using `severity>=INFO` while keeping the original exclusion filter on the default `_Default` bucket sink unchanged.