Question

Difficulty: Very hardAWS CloudFormation

A developer manages a production environment deployed using an AWS CloudFormation stack. The stack contains an Amazon RDS DB instance, an Amazon EC2 instance, and an IAM role. A system administrator manually changed the security group of the RDS DB instance in the Amazon VPC Console to address a transient connection issue.

During a subsequent stack update to upgrade the database engine version and add policies to the IAM role, the update fails during the RDS DB instance modification. CloudFormation attempts to roll back the changes, but the rollback fails and remains stuck in the `UPDATE_ROLLBACK_FAILED` state because the manually modified security group configuration prevents the database rollback. The developer must complete the stack update, ensuring the new IAM policies are applied and the database is upgraded.

Which sequence of actions must the developer perform to resolve this issue?

  1. A
    Execute `aws cloudformation update-stack` with the `--force` parameter to override the rollback state and directly apply the upgraded database engine version and IAM policies. Then, run drift detection on the stack and select the auto-reconcile option to automatically revert the manual security group changes.
  2. Execute `aws cloudformation continue-update-rollback` specifying the logical ID of the RDS DB instance in the `--resources-to-skip` parameter to transition the stack to `UPDATE_ROLLBACK_COMPLETE`. Revert the manual security group modifications on the RDS DB instance in the Amazon VPC Console to align the resource's physical state with the template, and then perform a new stack update with the corrected database engine version and the updated IAM role template.Answer
  3. C
    Execute `aws cloudformation rollback-stack` to force the stack state back to `ROLLBACK_COMPLETE`. Store the security group configuration as a secure string in AWS Systems Manager Parameter Store and reference it in the CloudFormation template using a dynamic reference, and then update the stack to apply the database and IAM role changes.
  4. D
    Manually delete the RDS DB instance from the Amazon RDS Console to trigger an automatic stack state change to `DELETE_COMPLETE`. Run `aws cloudformation continue-update-rollback` to complete the stack rollback, and then re-create the RDS DB instance manually and perform a stack update with a new template that references the new database instance ID.

Answer

Execute `aws cloudformation continue-update-rollback` with the `--resources-to-skip` parameter for the RDS DB instance, revert the manual security group modifications in the console, and then perform a new stack update.
The correct sequence begins by executing the `continue-update-rollback` command and specifying the logical ID of the RDS DB instance in the `--resources-to-skip` parameter. This instructs CloudFormation to set the status of the RDS DB instance to update/rollback complete (leaving its physical state unchanged) and proceed with rolling back the rest of the stack, shifting the overall stack status to `UPDATE_ROLLBACK_COMPLETE`. Because skipping the resource leaves it inconsistent with the template, the developer must manually revert the out-of-band security group changes in the VPC Console to match the template. Once reconciled, a new stack update can be run successfully to apply the database upgrade and the new IAM role policies.

Step-by-Step Solution

1
Invoke the continue-update-rollback command with skipped resources
The command `aws cloudformation continue-update-rollback --stack-name <stack-name> --resources-to-skip <RDS-Logical-ID>` is executed, transitioning the stack to the `UPDATE_ROLLBACK_COMPLETE` state.
When a stack is stuck in `UPDATE_ROLLBACK_FAILED`, you must use `continue-update-rollback`. Specifying the failing resource in `--resources-to-skip` allows CloudFormation to bypass rolling back that specific resource and successfully roll back the rest of the stack (such as the IAM role).
2
Reconcile resource drift manually
The manual changes made to the RDS DB instance security group are reverted in the AWS Management Console to match the configuration defined in the template.
Skipping a resource leaves its physical state inconsistent with the stack template. To avoid future update failures due to drift, the physical resource must be aligned with the template before launching a new update.
3
Perform the stack update again
The stack update is executed with the corrected configuration, upgrading the database engine version and applying the new IAM policies.
With the stack in a stable state (`UPDATE_ROLLBACK_COMPLETE`) and the resources reconciled, the update can now be cleanly processed.

Key Concept

Handling AWS CloudFormation UPDATE_ROLLBACK_FAILED states by skipping resources and reconciling out-of-band drift.
Estimated Time:3m 0s
Rate this question