CloudFormation Force Rollback 救援 UPDATE_ROLLBACK_FAILED 堆疊指南

There is one moment in day-to-day deployments that anyone who has worked with CloudFormation for a while knows: the update fails, the rollback starts, and then the stack just sits there with UPDATE_ROLLBACK_FAILED. No new deployment, no deletion from the pipeline, just a red status and the question of who changed “just a little thing in the console” yesterday. Whether you use CDK, the Serverless Framework or SAM, there is always a CloudFormation stack underneath, so this affects everyone.

On October 22, 2026, AWS introduced a new option for exactly this situation: Force Rollback. It gets a stack out of UPDATE_ROLLBACK_FAILED with a single call, even when several resources in nested stacks are stuck. That is convenient. It is also a tool that quietly lets drift creep into your infrastructure. In this article I show you step by step how I would handle a stuck stack, when Force Rollback is the right choice and what you absolutely should check afterwards.

In short

With continue-update-rollback --force-rollback, CloudFormation skips every resource whose rollback failed and brings the stack to UPDATE_ROLLBACK_COMPLETE in one step. The stack can be deployed again afterwards, but at the skipped spots it no longer matches the template. That is why a drift check with a drift-aware change set belongs before the next deployment.

What you will learn in this article

What UPDATE_ROLLBACK_FAILED means and which causes are typical
When to fix, skip selectively or use Force Rollback
How to run Force Rollback safely with the AWS CLI and a TypeScript script
What cdk rollback --force does differently from Force Rollback today
How to clean up drift with a drift-aware change set before the next deployment

What UPDATE_ROLLBACK_FAILED means

CloudFormation treats an update like a transaction: either all changes go through, or the stack is reset to the last working state. This reset is called a rollback. Most of the time it runs quietly, and at the end you see UPDATE_ROLLBACK_COMPLETE.

It gets problematic when CloudFormation cannot bring a resource back to its old state. The classic example: someone deleted a table or a database instance outside of CloudFormation. CloudFormation still believes it exists, tries to reconfigure it and fails. The stack ends up in UPDATE_ROLLBACK_FAILED. In this state it accepts no more updates. You have to finish the rollback before you can deploy again.

🎬 Status flow of a CloudFormation stack during a failed update – this animated diagram only runs on my site: watch it on sascha-ms.de

One thing matters for understanding this: UPDATE_ROLLBACK_COMPLETE does not mean “everything is as before”. It only means that CloudFormation considers the rollback finished. How well the state of the real resources matches the template depends on how you got there.

Typical causes of a stuck rollback

Before you skip anything, take a look at the cause. The CloudFormation documentation lists a handful of classics, and for most of them there is a clean fix that works without skipping:

Cause	What you can do
Resource was changed or deleted outside of CloudFormation	Restore the resource with the same name and properties, then continue the rollback
Missing IAM permissions for the role CloudFormation uses	Add the permissions, then continue the rollback
Service quota reached (limit)	Clean up resources you no longer need or request an increase
Expired security token	Simply continue the rollback, the credentials are refreshed in the process
Resource does not stabilize, e.g. during a service disruption	Wait, then continue the rollback
Not enough signals (e.g. with a CreationPolicy)	Send the missing signals with `signal-resource`

Only when none of these fixes works does skipping come into play. Until now, the only way was --resources-to-skip, where you had to list every affected resource by its logical ID. The logical ID is the name of a resource in the template; with CDK it is the generated identifier such as OrdersTable1A2B3C4D. Nested stacks added the format NestedStackName.LogicalId on top. Anyone who has had to piece that together for three levels of nested stacks under time pressure knows why AWS has followed up here.

What Force Rollback in CloudFormation does

Force Rollback is a new option of the existing ContinueUpdateRollback API call. In the SDK the parameter is called ForceRollback, in the AWS CLI --force-rollback. According to the AWS documentation it behaves like this:

Every resource stuck in UPDATE_FAILED due to a failed rollback gets skipped, and that also applies to nested stacks.
Resources that already failed during the actual update (before the rollback) are not skipped.
In the stack, skipped resources then show UPDATE_COMPLETE, with “Resource skipped during UpdateRollback” as the status reason.
Drift detection afterwards reports these resources as NOT_CHECKED.
--force-rollback and --resources-to-skip are mutually exclusive. If you pass both, you get a validation error.

The feature costs nothing extra and, according to the announcement, is available in all regions. What it does not do: repair your resources. It only makes CloudFormation stop clinging to them. The gap between template and reality remains, and you take care of it in the second part of this article.

My decision path: fix the cause first, then skip selectively or with Force Rollback, then check drift.

Step by step: getting a stack out of UPDATE_ROLLBACK_FAILED

Prerequisites

A current AWS CLI v2. If your CLI does not know the option, it reports Unknown options: --force-rollback. In my test, AWS CLI v1 version 1.46.2 did not know the parameter yet, while the JavaScript SDK @aws-sdk/client-cloudformation 3.1156.0 already did.
Permissions for cloudformation:DescribeStacks, DescribeStackEvents, ContinueUpdateRollback and, for the drift check, DetectStackResourceDrift plus read permissions on the affected resources.
For the script further down: Node.js 22 or newer.
Calm. Under pressure it is easy to copy the first command you find in a forum, and that is exactly what you want to avoid here.

I call the example stack OrdersStack in all commands. Replace the name with yours.

1. Check status and reason

aws cloudformation describe-stacks \
  --stack-name OrdersStack \
  --query "Stacks[0].[StackStatus,StackStatusReason]" \
  --output text

Expected result: UPDATE_ROLLBACK_FAILED and a reason such as “The following resource(s) failed to update: [OrdersTable]”. If you see a different status, you are dealing with a different problem, and this article will only help you to a limited extent.

2. Read the cause from the stack events

aws cloudformation describe-stack-events \
  --stack-name OrdersStack \
  --max-items 50 \
  --query "StackEvents[?ResourceStatus=='UPDATE_FAILED'].[Timestamp,LogicalResourceId,ResourceType,ResourceStatusReason]" \
  --output table

Events come newest first. The interesting ones are the UPDATE_FAILED entries after the rollback started, i.e. above the stack’s UPDATE_ROLLBACK_IN_PROGRESS event. The status reason column almost always tells you what is going on: “Resource not found”, “AccessDenied”, “LimitExceeded”.

3. Fix first, then continue normally

If you can fix the cause, for example by adding missing permissions or restoring a deleted resource with the same name, continue the rollback without options:

aws cloudformation continue-update-rollback --stack-name OrdersStack
aws cloudformation wait stack-rollback-complete --stack-name OrdersStack

This is the cleanest path, because template and reality match afterwards. The wait command blocks until the stack reaches UPDATE_ROLLBACK_COMPLETE and exits with an error if the rollback fails again.

4. Skip a single resource selectively

If exactly one resource is affected and you know you are deliberately giving it up, --resources-to-skip is still the more precise choice. You decide per resource instead of letting CloudFormation skip everything:

aws cloudformation continue-update-rollback \
  --stack-name OrdersStack \
  --resources-to-skip OrdersTable

5. Force Rollback when several resources are stuck

If several resources are stuck, possibly spread across several levels of nested stacks, Force Rollback saves you the search:

aws cloudformation continue-update-rollback \
  --stack-name OrdersStack \
  --force-rollback
aws cloudformation wait stack-rollback-complete --stack-name OrdersStack

Expected result: the stack is in UPDATE_ROLLBACK_COMPLETE, and the status reason points out that resources were skipped. You can deploy again. But you should not do it just yet.

6. Find the skipped resources

aws cloudformation describe-stack-events \
  --stack-name OrdersStack \
  --max-items 50 \
  --query "StackEvents[?contains(ResourceStatusReason || '', 'skipped during UpdateRollback')].LogicalResourceId" \
  --output text

With nested stacks, run the same command for the child stacks as well. You can find their names in the “Resources” tab of the root stack or in the stack ARN.

7. Check the drift of every skipped resource

Since drift detection lists skipped resources as NOT_CHECKED, you check them one by one. Drift means that the real configuration of a resource differs from what is in the template.

aws cloudformation detect-stack-resource-drift \
  --stack-name OrdersStack \
  --logical-resource-id OrdersTable \
  --query "StackResourceDrift.[StackResourceDriftStatus,PropertyDifferences]"

The result is IN_SYNC, MODIFIED or DELETED, and for MODIFIED it includes the differing properties. Now you know what needs to be sorted out before the next deployment.

The whole thing as a TypeScript script with the AWS SDK

If you need this more often or want it as a runbook tool for your team, a small script pays off. Mine has two modes: without a flag it is a dry run that only shows the status and the causes of the failure. Only with --force does it trigger the Force Rollback, wait for it to finish and check the drift of the skipped resources. This two-step approach is intentional. Whoever runs the script should have seen the causes before anything gets skipped.

Set up the project

mkdir stack-rescue && cd stack-rescue
npm init -y
npm pkg set type=module
npm install @aws-sdk/client-cloudformation
npm install -D typescript tsx @types/node

// tsconfig.json
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "noEmit": true,
    "skipLibCheck": true
  },
  "include": ["scripts", "test"]
}

The logic for evaluating the events

I put the evaluation of the stack events into a separate file with pure functions. That way you can test it without AWS access.

// scripts/rollback-events.ts
import type { StackEvent } from "@aws-sdk/client-cloudformation";

export const SKIP_REASON = "Resource skipped during UpdateRollback";

export interface FailedResource {
  logicalId: string;
  type: string;
  reason: string;
}

/**
 * Returns all resources whose rollback failed.
 * CloudFormation returns events newest first. We read up to the
 * latest UPDATE_ROLLBACK_IN_PROGRESS of the stack itself and collect
 * all UPDATE_FAILED events of resources along the way.
 */
export function rollbackFailures(events: StackEvent[], stackName: string): FailedResource[] {
  const result = new Map<string, FailedResource>();
  for (const e of events) {
    const isStackItself = e.LogicalResourceId === stackName && e.ResourceType === "AWS::CloudFormation::Stack";
    if (isStackItself && e.ResourceStatus === "UPDATE_ROLLBACK_IN_PROGRESS") break;
    if (isStackItself || e.ResourceStatus !== "UPDATE_FAILED" || !e.LogicalResourceId) continue;
    if (!result.has(e.LogicalResourceId)) {
      result.set(e.LogicalResourceId, {
        logicalId: e.LogicalResourceId,
        type: e.ResourceType ?? "unknown",
        reason: e.ResourceStatusReason ?? "",
      });
    }
  }
  return [...result.values()];
}

/** Resources that CloudFormation skipped during Force Rollback. */
export function skippedResources(events: StackEvent[], stackName: string): string[] {
  const ids = new Set<string>();
  for (const e of events) {
    const isStackItself = e.LogicalResourceId === stackName && e.ResourceType === "AWS::CloudFormation::Stack";
    if (isStackItself && e.ResourceStatus === "UPDATE_ROLLBACK_IN_PROGRESS") break;
    if (e.ResourceStatus === "UPDATE_COMPLETE" && e.ResourceStatusReason?.includes(SKIP_REASON) && e.LogicalResourceId) {
      ids.add(e.LogicalResourceId);
    }
  }
  return [...ids];
}

The actual script

// scripts/rescue-stack.ts
import { randomUUID } from "node:crypto";
import {
  CloudFormationClient,
  ContinueUpdateRollbackCommand,
  DescribeStacksCommand,
  DetectStackResourceDriftCommand,
  paginateDescribeStackEvents,
  waitUntilStackRollbackComplete,
  type StackEvent,
} from "@aws-sdk/client-cloudformation";
import { rollbackFailures, skippedResources } from "./rollback-events.js";

const [stackName, flag] = process.argv.slice(2);
if (!stackName) {
  console.error("Usage: npx tsx scripts/rescue-stack.ts <StackName> [--force]");
  process.exit(1);
}
const force = flag === "--force";
const cfn = new CloudFormationClient({});

async function loadEvents(maxPages = 5): Promise<StackEvent[]> {
  const events: StackEvent[] = [];
  let pages = 0;
  for await (const page of paginateDescribeStackEvents({ client: cfn }, { StackName: stackName })) {
    events.push(...(page.StackEvents ?? []));
    if (++pages >= maxPages) break;
  }
  return events;
}

const { Stacks } = await cfn.send(new DescribeStacksCommand({ StackName: stackName }));
const status = Stacks?.[0]?.StackStatus;
console.log(`Stack ${stackName}: ${status}`);
if (status !== "UPDATE_ROLLBACK_FAILED") {
  console.log("No Force Rollback needed.");
  process.exit(0);
}

const failures = rollbackFailures(await loadEvents(), stackName);
console.log("Rollback failed for:");
for (const f of failures) console.log(`- ${f.logicalId} (${f.type}): ${f.reason}`);

if (!force) {
  console.log("Dry run. Check the causes, then run again with --force.");
  process.exit(0);
}

await cfn.send(
  new ContinueUpdateRollbackCommand({
    StackName: stackName,
    ForceRollback: true,
    ClientRequestToken: randomUUID(),
  }),
);
console.log("Force Rollback started, waiting for UPDATE_ROLLBACK_COMPLETE ...");
await waitUntilStackRollbackComplete({ client: cfn, maxWaitTime: 1800 }, { StackName: stackName });

const skipped = skippedResources(await loadEvents(), stackName);
console.log(`Skipped: ${skipped.length ? skipped.join(", ") : "none (in the root stack)"}`);

for (const logicalId of skipped) {
  try {
    const drift = await cfn.send(
      new DetectStackResourceDriftCommand({ StackName: stackName, LogicalResourceId: logicalId }),
    );
    console.log(`Drift ${logicalId}: ${drift.StackResourceDrift?.StackResourceDriftStatus}`);
  } catch (err) {
    console.log(`Drift ${logicalId}: could not be checked (${(err as Error).name})`);
  }
}

Tests for the evaluation

// test/rollback-events.test.ts
import { test } from "node:test";
import assert from "node:assert/strict";
import type { StackEvent } from "@aws-sdk/client-cloudformation";
import { rollbackFailures, skippedResources, SKIP_REASON } from "../scripts/rollback-events.js";

const stack = "OrdersStack";
const ev = (id: string, type: string, status: string, reason?: string): StackEvent =>
  ({ LogicalResourceId: id, ResourceType: type, ResourceStatus: status, ResourceStatusReason: reason }) as StackEvent;

// Newest events first, as returned by DescribeStackEvents
const failedEvents: StackEvent[] = [
  ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_FAILED", "The following resource(s) failed to update: [OrdersTable]."),
  ev("OrdersTable", "AWS::DynamoDB::Table", "UPDATE_FAILED", "Resource not found"),
  ev("OrdersFn", "AWS::Lambda::Function", "UPDATE_COMPLETE"),
  ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_IN_PROGRESS"),
  ev("OrdersFn", "AWS::Lambda::Function", "UPDATE_FAILED", "forward update failed"),
];

test("finds only failures from the rollback phase", () => {
  const result = rollbackFailures(failedEvents, stack);
  assert.deepEqual(result.map((r) => r.logicalId), ["OrdersTable"]);
  assert.equal(result[0].reason, "Resource not found");
});

test("detects skipped resources by status reason", () => {
  const after: StackEvent[] = [
    ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_COMPLETE"),
    ev("OrdersTable", "AWS::DynamoDB::Table", "UPDATE_COMPLETE", SKIP_REASON),
    ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_IN_PROGRESS"),
    ...failedEvents,
  ];
  assert.deepEqual(skippedResources(after, stack), ["OrdersTable"]);
});

test("returns an empty list without rollback failures", () => {
  assert.deepEqual(rollbackFailures([ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_IN_PROGRESS")], stack), []);
});

Run it

# Type check and tests
npx tsc --noEmit
npx tsx --test test/*.test.ts

# Dry run: only shows status and failure causes
npx tsx scripts/rescue-stack.ts OrdersStack

# The real thing: Force Rollback, wait, check drift of skipped resources
npx tsx scripts/rescue-stack.ts OrdersStack --force

This is what the output might look like in a real case (example values):

Stack OrdersStack: UPDATE_ROLLBACK_FAILED
Rollback failed for:
- OrdersTable (AWS::DynamoDB::Table): Resource not found
Force Rollback started, waiting for UPDATE_ROLLBACK_COMPLETE ...
Skipped: OrdersTable
Drift OrdersTable: DELETED

The script deliberately looks only at the events of the root stack. For nested stacks, run it again with the name of the child stack to see the skipped resources there. I ran the type check and the three tests with Node.js 22.22.0, TypeScript 5.9.3 and SDK version 3.1156.0. I could not test it against a real stuck stack, simply because I did not have one at hand at that moment.

If you use the script in your team, give it its own narrowly scoped role. This policy covers the CloudFormation side; read permissions on the checked resources are needed in addition for the drift check:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "StackRescue",
      "Effect": "Allow",
      "Action": [
        "cloudformation:DescribeStacks",
        "cloudformation:DescribeStackEvents",
        "cloudformation:ContinueUpdateRollback",
        "cloudformation:DetectStackResourceDrift"
      ],
      "Resource": "arn:aws:cloudformation:eu-central-1:123456789012:stack/OrdersStack/*"
    }
  ]
}

What cdk rollback --force in AWS CDK does today

The CDK CLI has had its own rollback command for a while. You mainly need it when you work with cdk deploy --no-rollback, i.e. when you deliberately switch off automatic rollback to iterate faster:

# Roll back the stack normally (e.g. after cdk deploy --no-rollback)
npx cdk rollback OrdersStack

# Skip a single resource
npx cdk rollback OrdersStack --orphan OrdersTable

# Skip all failed resources
npx cdk rollback OrdersStack --force

According to the CDK documentation, --force is equivalent to setting --orphan for every resource. I wanted to know whether it already uses the new Force Rollback option internally and looked at the code of CDK CLI 2.1146.1. Result: not yet. The CLI reads the stack’s error events, collects the logical IDs of the failed resources and passes them as ResourcesToSkip. It only considers resources of the root stack, not those in nested stacks.

If you work without NestedStack, this makes no difference. But if you do have nested stacks, cdk rollback --force can get stuck at exactly the point that Force Rollback solves. In that case, call aws cloudformation continue-update-rollback --force-rollback directly on the stack that CDK already deployed. According to the AWS announcement, Force Rollback is also meant to be usable through CDK, so a later CLI version may well switch over. Check the release notes of your version if needed.

If CDK is new to you: in my article on Lambda Web Functions with CDK and CloudFront you can see a complete CDK project from start to finish.

Drift-aware change set before the next deployment

Now comes the part that is easy to skip after a stressful morning. AWS explicitly recommends creating a drift-aware change set after a Force Rollback, before you deploy the next time. A regular change set only compares the new template with the old one. A drift-aware change set compares three ways: the new template, the last deployment and the actual state of the resources. So it shows you whether your next deployment overwrites a manual change or recreates a deleted resource.

With the CDK CLI this takes three steps. The key is the combination of --method=prepare-change-set and --revert-drift: the change set is created but not executed yet.

# 1. Only create the drift-aware change set, do not execute it
npx cdk deploy OrdersStack \
  --method=prepare-change-set \
  --revert-drift \
  --change-set-name drift-check

# 2. Review the result
aws cloudformation describe-change-set \
  --stack-name OrdersStack \
  --change-set-name drift-check \
  --query "{Drift:StackDriftStatus,Changes:Changes[].ResourceChange.[LogicalResourceId,Action,ResourceDriftStatus]}"

# 3. All good? Then execute exactly this change set
npx cdk deploy OrdersStack \
  --method=execute-change-set \
  --change-set-name drift-check

Without CDK, for example with the Serverless Framework or your own template, you create the change set with the CLI. Replace the template path with your current template:

aws cloudformation create-change-set \
  --stack-name OrdersStack \
  --change-set-name drift-check \
  --template-body file://template.json \
  --deployment-mode REVERT_DRIFT \
  --capabilities CAPABILITY_IAM

There are two limitations you should know about. For some resource types CloudFormation does not support the comparison with the actual state and falls back to the classic comparison. And write-only properties such as passwords are compared against the last deployment, not against reality. The complete list is in the documentation on drift-aware change sets.

Common mistakes and how to avoid them

Force Rollback as the first reflex. If the cause was an expired token or missing permissions, a normal continue-update-rollback would have been enough, without any drift.
Combining both options. Using --force-rollback and --resources-to-skip together results in a validation error.
Deploying right away. The stack may be green, but the skipped resources do not match the template. The next update can fail at exactly that spot, and in the worst case the stack can no longer be recovered.
Targeting a nested stack directly. ContinueUpdateRollback expects the root stack. Specifying the child stack directly does not work.
Old CLI in the pipeline image. If the command works locally but fails in CI with “Unknown options”, the image is too old.
Not documenting the cause. Whoever intervened manually in the console will do it again. Write the cause into the incident log and talk about it in the team, without blame.

My take

I think Force Rollback is a good addition because it removes a really annoying chore. Especially with nested stacks, collecting the logical IDs was error-prone, and mistakes under time pressure are the most expensive ones. Still, for me the option is an emergency exit, not the standard path. It makes a stack usable again quickly, but it only postpones the actual work.

Much more important to me is the question behind it: why is the stack stuck in the first place? Very often the answer is a manual change to a resource that is supposed to be managed through infrastructure as code. It is no coincidence that the AWS documentation uses exactly that as its classic example. No CLI parameter fixes that. What is missing is a team agreement, or it exists but is not lived. If you need Force Rollback regularly, talk about permissions in production, regular drift detection and a clear rule for hotfixes rather than about the next CLI parameter.

That is why I see the real progress in the combination: Force Rollback gets you out of the dead end, and the drift-aware change set honestly shows you what is still off afterwards. If you use both together, you have a clean path from the red stack back to a state you can trust again.

One last thought: whether the real cause comes to light after an incident like this depends less on tooling than on the climate in the team. I describe how to make sure mistakes are discussed openly in my guide to psychological safety in teams.

FAQ

Can I simply delete a stack in UPDATE_ROLLBACK_FAILED?

Yes, a stack in this state can be deleted. But you lose all resources that are not protected by a DeletionPolicy. For stacks with data, going through the rollback is almost always the better path.

What is the difference between --force-rollback and --resources-to-skip?

With --resources-to-skip you name each resource individually, using the format NestedStackName.LogicalId for nested stacks. --force-rollback finds all resources with a failed rollback on its own and skips them, across all levels. Using both together is not allowed.

Does Force Rollback cost anything?

According to AWS, no. AWS charges nothing extra for the option, and it works in every region. Costs only arise from resources you create again afterwards.

Does this also work with the Serverless Framework?

Yes, because the Serverless Framework also deploys CloudFormation stacks. It has no dedicated command for this. You use the AWS CLI directly with the stack name the framework assigns.

Your next step

Put the commands from this article into your team runbook now, not during the next incident. Even better: try the procedure once in a sandbox account by deliberately deleting a resource manually and then provoking an update with a rollback. If you have walked the path once in peace, you stay calmer when it counts. If you are looking for a compact overview for your runbook, take a look at my cheat sheets.

And now I am curious: what caused the last stuck stack for you? And does your team have a clear rule about when someone may intervene manually in production?

Sources

AWS What’s New: AWS CloudFormation now simplifies stack update rollback operations (10/22/2026)
CloudFormation User Guide: Continue rolling back an update
CloudFormation User Guide: Troubleshooting – Update rollback failed
CloudFormation API Reference: ContinueUpdateRollback
CloudFormation User Guide: Drift-aware change sets
AWS CDK Developer Guide: cdk rollback
My own checks on 10/23/2026: @aws-sdk/client-cloudformation 3.1156.0 (parameter ForceRollback), AWS CLI v1 1.46.2 (parameter not included yet), source code of CDK CLI 2.1146.1 (cdk rollback --force, --revert-drift), type check and 3 tests with Node.js 22.22.0 and TypeScript 5.9.3

This post was created with AI assistance.

👉 Next step: Working with AWS CDK? Grab my printable AWS CDK cheat sheet with commands, concepts and a stack skeleton.

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en