A SysOps Administrator is troubleshooting a monitoring setup for a critical transactional API. A custom monitoring daemon on each Amazon EC2 instance in an Auto Scaling group publishes a custom metric named `TransactionLatency` to Amazon CloudWatch every 10 seconds. The metric is published with three dimensions: `InstanceId`, `AutoScalingGroupName`, and `Stage`. To detect latency spikes across the fleet, the administrator creates a CloudWatch metric alarm with a 10-second period, evaluating 3 datapoints, and utilizing the dimension `AutoScalingGroupName = production-asg`. However, the alarm constantly remains in the `INSUFFICIENT_DATA` state, even during peak periods when latency is high. What is the root cause of this issue?
- CloudWatch does not automatically aggregate custom metrics across dimensions. Since the metric is published with multiple dimensions including InstanceId, querying it using only the AutoScalingGroupName dimension returns no data.Answer
- BDetailed monitoring is not enabled for the EC2 instances in the Auto Scaling group, which restricts CloudWatch from evaluating metric alarms with a period of less than 5 minutes.
- CThe CloudWatch alarm is attempting to invoke a scaling policy directly, but high-resolution sub-minute alarms must route through Amazon EventBridge to execute Systems Manager Automation actions.
- DThe Auto Scaling group's cooldown period is set to 300 seconds, which overrides the alarm evaluation frequency and suppresses metrics to prevent scaling thrashing.
Answer
CloudWatch does not automatically aggregate custom metrics across dimensions. Since the metric is published with multiple dimensions including InstanceId, querying it using only the AutoScalingGroupName dimension returns no data.
CloudWatch custom metrics are uniquely identified by their complete set of dimensions. If a metric is published with multiple dimensions (such as InstanceId, AutoScalingGroupName, and Stage), querying or creating an alarm using only a subset of those dimensions (like AutoScalingGroupName) will fail to retrieve the data. CloudWatch does not perform automatic aggregation across dimensions for custom metrics. To alarm on the aggregate value, the metrics must be published with the desired target dimensions, or aggregated using CloudWatch Metric Math.
Step-by-Step Solution
Key Concept
Metric Dimensions and Custom Metric Aggregation in CloudWatch
Estimated Time:3m 0s