Examkin

Question

Difficulty: Very hardAWS CloudFormation

A developer is managing an application deployed via AWS CloudFormation. The application's database credentials are stored in AWS Secrets Manager and referenced in the CloudFormation template using the dynamic reference `{{resolve:secretsmanager:ProductionDBSecret:SecretString:Password}}` within the `UserData` property of an `AWS::EC2::LaunchTemplate` resource.

The database administrator rotates the database password in AWS Secrets Manager. Subsequently, the developer initiates a stack update to increase the `MaxSize` property of the `AWS::AutoScaling::AutoScalingGroup` resource that uses this launch template. The stack update completes successfully, but the newly launched EC2 instances fail to connect to the database, while existing instances continue to function until their cached credentials expire.

What is the cause of this issue, and how should the developer resolve it?

  1. A
    The developer should store the password in Systems Manager Parameter Store as a SecureString and use the `{{resolve:ssm-secure:ParameterName:Version}}` dynamic reference instead, as Parameter Store parameters automatically trigger a stack update and resource replacement whenever the parameter value changes.
  2. B
    The manual update of the database password created a drift condition between the stack template and the deployed resources, preventing CloudFormation from resolving the secret. The developer must run drift detection on the stack, resolve the drift, and perform a stack import.
  3. The Launch Template resource was not modified during the stack update, so CloudFormation did not re-resolve the dynamic reference to fetch the new password. The developer must update the template by appending the new secret's version ID or version stage to the dynamic reference to force a Launch Template update.Answer
  4. D
    The stack update failed during the Auto Scaling group scaling activity due to the incorrect credential, placing the stack in the `UPDATE_ROLLBACK_IN_PROGRESS` state. The developer must manually delete the newly created EC2 instances and use the `ContinueUpdateRollback` CLI command to force the stack back to `UPDATE_COMPLETE`.

Answer

The Launch Template resource was not modified during the stack update, so CloudFormation did not re-resolve the dynamic reference. The developer must update the template by appending the new secret's version ID or version stage to the dynamic reference to force a Launch Template update.
CloudFormation resolves dynamic references only during stack creation or when the specific resource containing the dynamic reference is updated. Since the stack update only modified the `MaxSize` property of the `AWS::AutoScaling::AutoScalingGroup` resource, the `AWS::EC2::LaunchTemplate` resource was not modified, and CloudFormation did not re-resolve the Secrets Manager dynamic reference. Consequently, the launch template continued to use the old resolved password value. To resolve this, the template must be updated to force an update of the launch template resource. Appending the new secret's version ID or version stage (such as the specific version UUID) to the dynamic reference changes the template definition of the launch template, forcing CloudFormation to update the launch template and retrieve the new secret value.

Step-by-Step Solution

1
Analyze why the new EC2 instances are failing to connect.
The new instances are launched using the launch template that still contains the old database password because the dynamic reference was not re-resolved.
CloudFormation only resolves dynamic references when the resource defining them is created or updated. The stack update only changed the Auto Scaling group's MaxSize, leaving the Launch Template untouched.
2
Determine the correct method to force CloudFormation to resolve the updated secret.
The Launch Template resource must be modified in the template so that CloudFormation initiates an update for it.
By changing the resource's definition, CloudFormation is triggered to update the Launch Template and fetch the updated value of the dynamic reference.
3
Apply the change using the Secrets Manager dynamic reference version ID/stage suffix.
Update the dynamic reference to specify the new version ID (e.g., `{{resolve:secretsmanager:ProductionDBSecret:SecretString:Password::version-id}}`).
This updates the launch template resource in the CloudFormation template, prompting CloudFormation to re-resolve the reference during the stack update.

Key Concept

CloudFormation dynamic references are only resolved during resource creation or updates. Changing properties of other resources (like Auto Scaling group size) does not trigger re-resolution of dynamic references in unchanged resources.

Alternative Method

Another way to force the launch template to update and re-resolve the dynamic reference is to update a non-disruptive parameter or property on the launch template itself, such as adding or changing a metadata property or changing the launch template version in the Auto Scaling group template definition.
Estimated Time:3m 0s
Rate this question

Topics

AWS CloudFormation

More questions from this topic