A financial application deployed on a GKE Standard cluster relies on a Horizontal Pod Autoscaler (HPA) to scale Pod replicas from 4 to 40 during high-traffic trading hours. During a recent market volatility event, HPA triggered scaling, but newly created web Pods remained stuck in a `Pending` state for over 15 minutes due to insufficient CPU capacity on existing cluster nodes. Additionally, the operations team plans to run batch analytics workloads on a dedicated cost-optimized Spot VM node pool, but must prevent latency-sensitive web Pods from being scheduled on these Spot nodes. Which combination of operations correctly resolves the node starvation issue and isolates the batch workloads?
- Enable Cluster Autoscaler on the web workload node pool using `gcloud container node-pools update --enable-autoscaling`, then add a taint to the Spot VM node pool and configure matching tolerations on the batch workload Pod manifests.Answer
- BUpdate the Horizontal Pod Autoscaler manifest using `kubectl edit hpa` to increase the maximum replicas limit to 100, and remove all node taints from the Spot VM node pool to allow automatic overflow.
- CConfigure the web Pod spec with nodeAffinity pointing to the Spot VM node pool while relying on HPA to scale node count when Pod CPU utilization exceeds 80%.
- DRun `gcloud container clusters update` with the `--enable-autoscaling` flag on the cluster and migrate the cluster in-place to GKE Autopilot without configuring node pool taints.
Answer
Enable Cluster Autoscaler on the web workload node pool using gcloud container node-pools update --enable-autoscaling, add a taint to the Spot VM node pool, and specify matching tolerations on the batch job Pod manifests.
When Pods remain in a Pending state due to lack of cluster CPU, infrastructure-level autoscaling is required. Enabling Cluster Autoscaler on the GKE node pool allows GKE to automatically resize Compute Engine nodes. Applying taints to the Spot node pool and adding matching tolerations to batch workloads ensures that latency-sensitive microservices are never scheduled on interruptible Spot instances.
Step-by-Step Solution
Key Concept
Dual-layer autoscaling (HPA for Pods, Cluster Autoscaler for Nodes) combined with node taints and tolerations for workload isolation.