Examkin

Question

Difficulty: HardImproving Operational Excellence via Monitoring and Logging

An enterprise runs a critical application on Amazon EC2 instances in an Auto Scaling group across multiple AWS accounts. The application writes log events to a local file at `/var/log/app/production.log`. A log rotation utility runs hourly, renaming the file to `/var/log/app/production.log.YYYY-MM-DD-HH` and creating a new empty `/var/log/app/production.log` file. Currently, the Unified CloudWatch Agent is installed on the EC2 instances, and its configuration file specifies `/var/log/app/production.log` as the log source. The operations team reports that log events written immediately before and during the hourly rotation are frequently missing from Amazon CloudWatch Logs. Additionally, the company's security policy requires all application logs to be consolidated into a single Amazon S3 bucket located in a dedicated Security account for long-term retention. Which combination of actions should the Solutions Architect recommend to resolve the log loss and establish the centralized logging pipeline?

  1. Update the CloudWatch Agent configuration on the EC2 instances by setting the `file_path` parameter to `/var/log/app/production.log*`. Configure a CloudWatch Logs subscription filter in each application account to stream the logs to an Amazon Kinesis Data Firehose delivery stream in the same account. Configure each Kinesis Data Firehose stream to deliver the logs to the centralized S3 bucket in the Security account, and update the S3 bucket policy in the Security account to grant `s3:PutObject` permissions to the IAM role principal of the Kinesis Data Firehose stream from each application account.Answer
  2. B
    Keep the CloudWatch Agent configuration's `file_path` set to `/var/log/app/production.log` to maintain static tracking. Configure a cron job on the EC2 instances to copy the active log file to a backup directory 5 minutes before the log rotation utility executes, and set up a secondary CloudWatch agent process to upload logs from the backup directory. Create a cross-account Kinesis Data Firehose stream to collect logs from CloudWatch and store them in the centralized S3 bucket.
  3. C
    Update the CloudWatch Agent configuration on the EC2 instances by setting the `file_path` parameter to `/var/log/app/production.log*`. Configure the CloudWatch Agent to directly write the logs to the centralized S3 bucket in the Security account. Enable default S3 bucket encryption using the AWS-managed KMS key for S3 (`aws/s3`) in the Security account, and configure the application account IAM roles to assume a role in the Security account to perform the uploads.
  4. D
    Update the CloudWatch Agent configuration on the EC2 instances by setting the `file_path` parameter to `/var/log/app/production.log*`. Configure CloudWatch Logs in the application accounts to stream logs directly to the centralized S3 bucket in the Security account. Update the S3 bucket policy in the Security account to grant `s3:PutObject` permissions, specifying the application AWS account IDs as the IAM principals in the principal element of the statement.

Answer

Update the CloudWatch Agent configuration on the EC2 instances to use a wildcard (`/var/log/app/production.log*`) for the `file_path` parameter. Establish a CloudWatch Logs subscription filter in each application account to send logs to Kinesis Data Firehose, and configure Firehose to deliver logs to the centralized S3 bucket in the Security account while updating the S3 bucket policy to grant `s3:PutObject` permissions to the Firehose IAM role principal from each application account.
The correct option addresses the log rotation issue by using a wildcard pattern (`/var/log/app/production.log*`) in the CloudWatch agent's `file_path` configuration. This ensures that the agent continues to read the rotated log files until they are fully consumed. Centralizing these logs is achieved by utilizing CloudWatch Logs subscription filters to send data to Kinesis Data Firehose in each application account, which then writes to the central S3 bucket in the Security account. The S3 bucket policy is updated to explicitly allow the Kinesis Data Firehose IAM roles from the application accounts to write (`s3:PutObject`) to the bucket, which is the standard, secure way to enable cross-account delivery.

Step-by-Step Solution

1
Configure the CloudWatch Agent with a wildcard in the file path.
The CloudWatch Agent tracks both the active log file and any rotated log files matching the pattern.
When log files are rotated, the active file is renamed (e.g., `production.log.YYYY-MM-DD-HH`). Using a static path causes the agent to stop tailing the rotated file, resulting in missing log entries that had not yet been processed at the moment of rotation. A wildcard pattern ensures the agent tails the rotated file to completion.
2
Set up CloudWatch Logs subscription filters to stream logs to Amazon Kinesis Data Firehose.
Logs are automatically streamed in near real-time from CloudWatch Logs to the Firehose delivery stream.
Streaming logs from CloudWatch Logs via subscription filters is a robust and scalable pattern to export log data without manual intervention or batch scripts.
3
Configure Kinesis Data Firehose to write to the centralized S3 bucket and configure the cross-account bucket policy.
Kinesis Data Firehose delivers log files to the centralized S3 bucket in the Security account securely.
To support cross-account log delivery, the Kinesis Data Firehose IAM role in the application account must be granted permission in the Security account's S3 bucket policy using the role's ARN as the principal.

Key Concept

Log file rotation tracking and cross-account log centralisation
Rate this question

Topics

Improving Operational Excellence via Monitoring and Logging