Question

Difficulty: HardDeployment Strategies and Execution

A SysOps Administrator uses AWS CloudFormation to manage a production infrastructure stack. The deployment is executed using a dedicated CloudFormation service role. The Administrator initiates a stack update to deploy a new version of an Amazon ECS service, which references a newly created ECS Task Execution IAM Role. During the deployment, the stack update fails because the service cannot start the new tasks, and CloudFormation transitions to UPDATE_ROLLBACK_IN_PROGRESS. However, the rollback operation also fails, and the stack ends up in the UPDATE_ROLLBACK_FAILED state. Upon reviewing the CloudFormation events, the Administrator discovers that the rollback failed because CloudFormation was unauthorized to associate the original ECS Task Execution Role back to the ECS service. Which of the following is the root cause of this failure, and how should it be resolved?

  1. The CloudFormation service role lacks the iam:PassRole permission for the original ECS Task Execution Role. To resolve this, attach an IAM policy granting the iam:PassRole permission for the original role to the CloudFormation service role, and then select the Continue update rollback action in the CloudFormation console.Answer
  2. B
    The Administrator's IAM user policy lacks the iam:PassRole permission for the original ECS Task Execution Role. To resolve this, add the iam:PassRole permission for the original role to the Administrator's IAM user, and then initiate a new stack update targeting the previous template.
  3. C
    The ECS cluster does not have sufficient capacity to run the original task definition. To resolve this, manually scale up the ECS cluster's container instances, and then click Update Stack to force a redeployment of the original template.
  4. D
    The CloudFormation service role lacks the sts:AssumeRole trust relationship with the ECS service principal. To resolve this, edit the trust policy of the CloudFormation service role to allow ecs-tasks.amazonaws.com to assume it, and then delete the failed stack to redeploy it from scratch.

Answer

The root cause is that the CloudFormation service role lacks the iam:PassRole permission for the original ECS Task Execution Role. The solution is to attach a policy granting this permission to the CloudFormation service role and choose Continue update rollback in the console.
The correct answer correctly identifies that the CloudFormation service role requires the iam:PassRole permission for the original ECS Task Execution Role to successfully revert the service configuration during rollback. Applying this permission to the service role and initiating 'Continue update rollback' allows the stack to successfully revert to its previous stable state.

Step-by-Step Solution

1
Analyze the CloudFormation stack state and the specific authorization error in the stack events.
Identify that the rollback failed (UPDATE_ROLLBACK_FAILED) specifically due to lack of authorization to associate the original ECS Task Execution Role.
To determine whether the issue lies with service-level execution permissions (the CloudFormation service role) or user permissions.
2
Update the CloudFormation service role permissions.
Attach an IAM policy allowing iam:PassRole on the original ECS Task Execution Role ARN to the CloudFormation service role.
When CloudFormation updates or rolls back an ECS service to use an IAM role, the CloudFormation service role must have explicit permission to pass that role to the ECS service.
3
Resume the failed rollback operation in the CloudFormation console.
Execute the 'Continue update rollback' action on the stack.
Stacks in UPDATE_ROLLBACK_FAILED cannot be updated or modified until the rollback is successfully completed or bypassed.

Key Concept

Troubleshooting CloudFormation rollback failures caused by missing service role permissions (specifically iam:PassRole) during ECS service updates.
Rate this question