模型備援鏈為何會悄然失效

The fallback chain that silently stops working

I found out my fallback chain was broken because of a bill. Not an alert, not a red row in a dashboard — the spend for that week came out lower than usual in a way that felt wrong, and when I went looking, every request had been answered by the last model in the chain. The app was fine. Latency was tolerable. Nobody complained. I had been running on the emergency route for a while and calling it production.

The first route in the chain had been retired. What stayed with me afterwards was how loudly that should have failed. A retired OpenAI model answers with HTTP 404 and not_found_error / model_not_found. That is a hard refusal, not a wobbly timeout. My chain caught that refusal, fell through to the next route, and kept going — which is precisely what I built it to do. By the time the error reached me it had already been swallowed and re-labeled as normal traffic.

The chain hides the failure it was built to survive

A fallback chain is a bet that a failing route is a temporary event. When that bet is right, you get a short blip and a slightly different answer. When the route is gone for good, the chain still does the right-looking thing: it serves the request from whatever is left. A chain with one surviving member is a green system. It is also a system you no longer understand, because the thing that broke is the thing that was supposed to protect you from breakage.

The uncomfortable part is that a fallback which fires for a good reason and a fallback which fires because your primary route was deleted look identical from the outside. Both produce a successful response. The only difference is which model wrote it, and unless you log that, you have no signal at all. I didn't log it. So for a while my only instrument was the invoice.

Retirement is announced, if you read the field that carries it

Retirement is not a mystery; it is data you have to go and fetch. OpenAI's model list exposes the schedule: GET /v1/models and the single-model "Retrieve a model" call carry an optional shutdown_date field, either a YYYY-MM-DD or null. The published notice periods are generous on paper — generally available models get six months or more, specialized variants at least three months, preview models sometimes only about two weeks — and fine-tunes follow the shutdown schedule of the base model they were trained from. That last clause is the one that bites, because your fine-tune keeps its own name and inherits someone else's deadline.

Anthropic frames the same thing as a lifecycle: Active, Legacy, Deprecated, Retired, with the plain statement that requests to retired models will fail. There, the guarantee is a minimum notice window that applies to customers with active deployments. Read that qualifier carefully. If your traffic to a model has gone quiet — because a fallback quietly took over, say — you are much less of an active deployment than you think you are, and the announcement you were counting on may not arrive.

The quieter breakages: parameters, credits, caps

Retirement is only the loudest way a route dies. Two others cut a route without ever saying the word "deprecated". The first is the parameter surface. On Claude 4.7 and later, passing temperature, top_p or top_k with a non-default value produces HTTP 400. My request builder had been sending a temperature for years because it once made the output nicer; on a newer model that same habit stops the call dead, and if it sits behind a fallback, the route is simply gone.

The second is credit and cap accounting. Anthropic's usage credits expire one calendar year after they are issued, are not refundable, and the expiry date cannot be extended; closing the account destroys whatever balance is left. On the traffic side, the error shapes matter as much as the messages. A rate_limit_error at 429 is also what you get when a tier spend cap is reached — with no retry-after, and it stays that way until access comes back. Self-set spend limits surface as 400. A billing problem arrives as 402 billing_error. None of these are "the model is gone", and all of them can take a route out of service for longer than a retired model would.

Ping every route on its own, at max_tokens=1

The test that would have caught my outage is almost insultingly small: address each route directly, one at a time, with max_tokens=1, and never through the chain. One token is enough to get an HTTP status and an error body, and it is cheap enough to run on a schedule without thinking about it.

curl -sS -o /tmp/route.json -w '%{http_code}\n' \
  https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"<the exact model string this chain entry uses>","max_tokens":1,"messages":[{"role":"user","content":"ping"}]}'

The point is what you do with the answer. If the body carries not_found_error / model_not_found, the route is not flaky, it is gone, and no amount of retry logic will bring it back. If it is a 400 on a parameter you thought was harmless, that is a code fix, not an incident. If it is 429 or 402, the route is fine and your wallet is the problem. Three different illnesses, three different responses — and a chain that answers every one of them the same way is not a monitoring strategy.

Don't trust the monitor; compare against the live map

My uptime check was pointed at the chain. That was the mistake. A health check that goes through a chain with one live route is green by construction, forever, which means it measures the chain's willingness to answer rather than the health of any route inside it. I had a monitor that could never fail and I read it every morning.

What replaced it is duller. The chain lives in a file as data, not as nested if branches. A job fetches the provider's model list, reads shutdown_date, and diffs that list against the chain. Any chain entry whose name no longer appears, or whose shutdown date has passed, gets flagged before a request ever touches it. Anything running traffic that Anthropic has marked deprecated is findable the same way the vendor suggests you find it: Console → Usage → Export, the CSV broken out by API key and model, which shows which deprecated models are still being called and by which key. I also log which route actually answered each request, because the shift from "primary" to "last resort" is a symptom, and it is invisible in every dashboard that only counts successes.

The number that finally told me something was wrong was the bill. I would rather it be the diff next time.