Question

Difficulty: MediumMonitoring and Analyzing Logs with Amazon CloudWatch

An application deployed on AWS Fargate publishes structured JSON logs to an Amazon CloudWatch Logs log group. Each log event contains fields such as `latency`, `statusCode`, `path`, and `userId`. A developer is tasked with creating a CloudWatch Logs Insights query to analyze application performance. The query must calculate the 95th95\text{th} percentile of latency for all requests and count the number of server errors (where `statusCode` is 500500 or greater). The results must be grouped by the API `path` and aggregated into 55-minute intervals. Which CloudWatch Logs Insights query should the developer use to meet these requirements?

  1. A
    fields @timestamp, path, latency, statusCode | filter statusCode >= 500 | stats pct(latency, 95) as p95_latency, count() as error_count by path, bin(5m)
  2. B
    fields @timestamp, path, latency, statusCode | stats pct(latency, 95) as p95_latency, count(statusCode >= 500) as error_count by path, bin(5m)
  3. fields @timestamp, path, latency, statusCode | stats pct(latency, 95) as p95_latency, sum(statusCode >= 500) as error_count by path, bin(5m)Answer
  4. D
    fields @timestamp, path, latency, statusCode | stats pct(latency, 95) as p95_latency, count() filter(where statusCode >= 500) as error_count by path, bin(5m)

Answer

The query that uses the sum function with a conditional expression inside stats: 'fields @timestamp, path, latency, statusCode | stats pct(latency, 95) as p95_latency, sum(statusCode >= 500) as error_count by path, bin(5m)'
The correct query uses sum(statusCode >= 500) inside the stats command. In CloudWatch Logs Insights, boolean expressions inside aggregation functions evaluate to 1 for true and 0 for false. Therefore, summing the expression statusCode >= 500 effectively counts only the events where the status code indicates a server error, while allowing the percentile function pct(latency, 95) to be calculated over the entire dataset without prior filtering.

Step-by-Step Solution

1
Determine where to place the filtering/conditional logic to ensure latency is calculated over all requests.
Avoid using a top-level '| filter statusCode >= 500' command, as it would prematurely discard successful requests before the latency calculation.
A top-level filter restricts the input dataset to only matching records, skewing overall metrics like latency percentiles.
2
Select the correct conditional aggregation function in CloudWatch Logs Insights.
Use 'sum(statusCode >= 500)' to sum the boolean results (1 for true, 0 for false).
Boolean expressions inside 'sum()' evaluate to 1 when true and 0 when false, which acts as a conditional count.
3
Group and bin the aggregated results.
Use 'by path, bin(5m)' at the end of the 'stats' command.
This groups the calculated statistics by the request path and segments them into 5-minute time intervals.

Key Concept

Conditional aggregation in CloudWatch Logs Insights stats command
Estimated Time:1m 30s
Rate this question