An enterprise SaaS provider hosts an event-driven data ingestion service on a Compute Engine Managed Instance Group (MIG). The service pulls messages from a Cloud Pub/Sub topic and writes structured output to a database. During peak periods, the queue length in Pub/Sub increases significantly, causing data processing latencies that breach Service Level Objectives (SLOs), even though average VM CPU utilization stays below 35% because the workload is heavily I/O-bound. Furthermore, a scheduled global marketing campaign next month is projected to increase incoming message volume by 400%. Which strategy should a Cloud Architect implement to ensure operational reliability and adequate capacity during peak workloads and the upcoming event?
- Configure target-tracking autoscaling for the MIG based on a custom Cloud Monitoring metric for Cloud Pub/Sub unacknowledged message count, and proactively request a Compute Engine regional CPU quota increase prior to the marketing campaign.Answer
- BLower the target CPU utilization threshold on the MIG autoscaler from 70% to 20% to force the cluster to scale up aggressively during queue processing spikes.
- CMigrate the message ingestion instances to a Google Kubernetes Engine (GKE) cluster with a Horizontal Pod Autoscaler (HPA) configured on standard CPU metrics to automatically handle dynamic queue processing.
- DEnable predictive autoscaling on the Compute Engine MIG based on historical CPU trends, relying on GCP's automatic quota burst features to accommodate the marketing campaign.
Answer
The optimal solution is to scale the Compute Engine Managed Instance Group using a custom Cloud Monitoring metric representing Cloud Pub/Sub queue depth, while requesting regional resource quota increases in advance of anticipated high-volume traffic events.
For I/O-bound queue processing microservices, CPU utilization is insufficient as a scaling trigger because instances spend significant time waiting on external calls while message backlogs accumulate. Utilizing Cloud Monitoring custom metrics based on Pub/Sub unacknowledged message depth allows the Managed Instance Group to scale precisely based on outstanding work. Additionally, preparing for major planned traffic increases requires auditing project quotas and requesting regional Compute Engine quota increases in advance, as GCP hard quotas do not scale automatically without explicit requests.
Step-by-Step Solution
Key Concept
Auto-scaling metric selection for I/O-bound queue workers and proactive capacity quota management
Estimated Time:2m 0s