如何在 AI 編寫的 UI 面向所有用戶前，以功能旗標控制發布

The first time we shipped an agent-written settings panel straight to main, it looked fine in staging. Then support got three tickets in an hour: a toggle that did nothing for free plans, a “Save” that double-fired on slow networks, and a new empty state that hid the only CTA half our users needed.

CI was green. Visuals looked close. The bug was product judgment the model never had — and we gave 100% of traffic that judgment on day one.

So we stopped treating “agent UI merged” as “agent UI live.” Feature flags became the default release valve for anything an agent touched in the UI layer.

The failure mode: merge = rollout

Agent PRs tempt you to skip the usual caution:

• Diff is “just UI”
• Screenshots look right
• Playwright clicked through the happy path
• Reviewer approved the code shape

None of that answers: does this layout work for the cohort that never sees premium fields? Does the new empty state break the onboarding funnel? Does the regenerated form still respect the old keyboard path power users rely on?

Those are traffic questions. Feature flags are how you ask them without a full revert.

What I flag (and what I don’t)

I flag user-visible surfaces the agent authored or heavily rewrote:

• New panels, modals, empty / error / loading states
• Regenerated forms and settings sections
• Agent-refactored navigation / IA changes
• Copy-heavy UI where tone or CTA placement shifted

I usually don’t flag:

• Pure token / lint / type cleanup with no layout change
• Test-only or Storybook-only diffs
• Internal admin tools with a tiny known audience (still stage them; just don’t burn a flag name)

Rule of thumb: if a stranger on the free plan could hit it tomorrow, it gets a flag.

The default pattern we use

Keep it boring. One boolean (or percentage) per agent UI surface, named after the ticket, not the model:

// flags.ts — names are product, not “gpt-panel-v3”
export const flags = {
  settingsBillingV2: booleanFlag('settings-billing-v2', false),
  emptyProjectsAgentCopy: booleanFlag('empty-projects-agent-copy', false),
} as const;

In the route / component:

export function SettingsBilling() {
  const enabled = useFlag('settings-billing-v2');

  if (!enabled) {
    return <SettingsBillingLegacy />;
  }

  return <SettingsBillingV2 />; // agent-authored surface
}

Kill switch lives next to the flag definition, not in a Slack thread:

// ops: set settings-billing-v2 = false → instant revert without redeploy

If your stack is LaunchDarkly / Unleash / Statsig / a homegrown % cookie — same idea. The important part is default off for agent UI, not which vendor.

Rollout ladder (what we actually do)

1. Internal / dogfood — flag on for the team only. Click the weird paths humans use.
2. 1–5% — watch error rate, rage clicks, support tags for that surface.
3. 25% → 50% — only if step 2 is quiet for a real workday, not twenty quiet minutes.
4. 100% — then delete the legacy branch when you’re sure, not “someday.”

Agent UI fails differently than human UI: it often fails narrowly (one plan tier, one locale, one empty-state branch). Small percentages catch that before it becomes a Monday war room.

What I put in the agent prompt

Feature flags are a release policy. The agent still needs to build behind them:

Implement this UI behind flag settings-billing-v2 (default off).

Keep SettingsBillingLegacy working and unchanged.

Do not delete legacy routes/components in this PR.

Wire the flag at the route/section boundary, not inside every leaf.

Add a short “flag off = legacy” note in the PR body.

Without that, agents “helpfully” delete the old panel and leave you with a binary deploy rollback.

PR template checkboxes

### Agent UI rollout
- [ ] New/changed user-visible UI is behind a named flag (default off)
- [ ] Legacy path still renders when flag is off
- [ ] Flag name matches the ticket / surface (not the model name)
- [ ] Rollout plan noted: dogfood → % → 100% (or “internal-only”)
- [ ] Kill switch owner named (who can flip it in prod)

Reviewers stop arguing “is this ready?” and start asking “who turns it on?”

Metrics I watch for the first day

Not vanity:

• Error / exception rate on the flagged route
• Support tickets tagged with that surface
• Funnel drop at the new empty / CTA step
• Accessibility smoke (keyboard + one screen reader pass on dogfood)

If any of those spike at 5%, you don’t “monitor overnight.” You flip the flag off and open a fix PR.

Review comments I leave

When the agent rewrote a panel with no flag:

This surface ships to every user on merge. Wrap it in settings-billing-v2 (default off) and keep legacy until we ramp. Agent UI doesn’t get a free pass on rollout.

When they deleted legacy in the same PR:

Don’t delete SettingsBillingLegacy here. Flag-off must still work. Removal is a follow-up after 100% + quiet metrics.

When the flag is hardcoded true “for testing”:

Hardcoded true is a silent 100% rollout. Use the real flag client; leave default false in prod config.

What I stopped doing

• Merging agent UI straight to 100% because staging looked pretty
• Naming flags after models (gpt-settings-rewrite) instead of product surfaces
• Letting the agent delete legacy in the same PR that introduces the new UI
• Treating “Playwright green” as “safe for everyone”

Feature-flagging AI-suggested UI isn’t distrust of the model — it’s admitting the model never saw your free-plan edge cases, your support queue, or your Monday traffic. Ship the code when it’s ready. Ship the exposure when the numbers say so.