2026 年如何建立能應對大語言模型 API 錯誤的 MCP 伺服器

Building an MCP server that survives LLM API errors in 2026

Classify the error before you retry it. Check the status code and the error type in the body, then look for any delay the server asked for (retry-after, OpenAI's reset headers, Gemini's RetryInfo). Retry 429s and 5xx with jittered backoff that never undercuts that delay. Fail fast on quota and auth errors, and give the MCP client a readable message instead of a traceback. The Python server below is about 90 lines.

The AI API Error Decoder I link to below is one I built. I'd gone through four different status-code cheat sheets, and every one of them said a 429 means "slow down and retry," which is wrong for OpenAI's insufficient_quota. It's free, it runs client-side, it doesn't need a signup, and nothing you paste is uploaded anywhere. If you have a better one, tell me.

The goal: a tool call that fails politely

Here's the setup that broke on me. In September 2026 I had a research agent running in Claude Code, and it called a small MCP server of mine. That server calls the Anthropic API itself, so a model is calling a tool that calls a model. It's turtles all the way down, and it worked fine until the night it didn't.

At 2:14am Anthropic returned a burst of 529 overloaded_error. My tool didn't handle it, so the MCP client got a raw httpx traceback with forty lines of stack frames. The agent read that traceback, decided the tool was broken, and stopped calling it for the rest of the session. I had queued 1,247 documents. It finished 312.

The model's reaction was reasonable. When a tool hands an agent a stack trace, the agent can't distinguish "try again in ten seconds" from "this will never work," so it gives up.

So the server we're building does one narrow job. It exposes a summarize tool backed by Claude. It retries the failures that are worth retrying, it stops right away on the ones that aren't, and every error reaching the client is one line long and includes a request ID. If the outage keeps going, the client shows something like 529 overloaded_error: Overloaded (request-id req_011CU...) after four quiet retries. If the key is bad, the client shows 401 authentication_error: invalid x-api-key on the first attempt, with no waiting.

Setup and auth (and the env var trap)

You need Python 3.10 or newer, because the code uses str | None type hints. You also need uv, which lets the script declare its dependencies on the command line, so there's no virtualenv to manage.

Get an Anthropic key from the console and register the server with Claude Code:

claude mcp add llm-guard -e ANTHROPIC_API_KEY=sk-ant-... -- uv run --with "mcp[cli]" --with httpx /absolute/path/llm_guard.py

The env var is the first gotcha. MCP clients launch stdio servers as child processes, and Claude Desktop in particular doesn't load your shell profile. Exporting ANTHROPIC_API_KEY in .zshrc won't help. The server starts and the first request dies with a KeyError, which (you guessed it) shows up as a traceback. Pass the key explicitly through -e or through the env block in the client's JSON config.

The second gotcha is the anthropic-version header. The Messages API requires it, and if you leave it out you get a 400 invalid_request_error with a message that's clear enough once you read it. I lost about ten minutes because my code treated every 400 as "your prompt is bad," and that sent me looking in the wrong place.

That second gotcha is the reason I keep the AI API Error Decoder open in a tab while writing this kind of code. When an error body showed up that I didn't recognize, I pasted it in to see which bucket it belonged to before writing a branch for it. Error bodies from Claude, OpenAI, Gemini, and Mistral look similar enough to fool you and differ enough to break your parser.

The core code

This is the entire server. Save it as llm_guard.py. The comments mark the spots that bit me.

# llm_guard.py
# Run: uv run --with "mcp[cli]" --with httpx llm_guard.py
import asyncio
import logging
import os
import random
import re
import sys

import httpx
from mcp.server.fastmcp import FastMCP

# stdio MCP servers speak JSON-RPC over stdout. Log to stderr or you corrupt the stream.
logging.basicConfig(stream=sys.stderr, level=logging.INFO)
log = logging.getLogger("llm-guard")

mcp = FastMCP("llm-guard")
ANTHROPIC_URL = "https://api.anthropic.com/v1/messages"
RETRYABLE = {429, 500, 502, 503, 504, 529}


def parse_duration(value: str | None) -> float | None:
    """Accepts '20', '37s', '1.5s', '500ms', or OpenAI-style '6m0s'."""
    if not value:
        return None
    if re.fullmatch(r"\d+(\.\d+)?", value):
        return float(value)
    units = {"ms": 0.001, "s": 1, "m": 60, "h": 3600}
    total = sum(float(n) * units[u] for n, u in re.findall(r"(\d+(?:\.\d+)?)(ms|s|m|h)", value))
    return total or None


def classify(status: int, headers: httpx.Headers, body: dict) -> dict:
    err = body.get("error", body)  # some providers return a flat body
    if isinstance(err, str):
        err = {"message": err}
    kind = err.get("type") or err.get("status") or err.get("code") or "unknown"
    msg = err.get("message", "")

    # Server-supplied delay. Header first, then Gemini's RetryInfo in the body.
    wait = parse_duration(headers.get("retry-after"))
    for detail in err.get("details") or []:
        if isinstance(detail, dict) and "retryDelay" in detail:
            wait = parse_duration(detail["retryDelay"])

    # OpenAI sends empty-billing as a 429. Retrying it just burns wall-clock time.
    if "insufficient_quota" in (err.get("type"), err.get("code")):
        return {"retry": False, "wait": None, "why": f"{status} insufficient_quota: add credits"}
    if status in RETRYABLE:
        return {"retry": True, "wait": wait, "why": f"{status} {kind}: {msg}"}
    return {"retry": False, "wait": None, "why": f"{status} {kind}: {msg}"}


async def call_claude(prompt: str, max_attempts: int = 5) -> str:
    headers = {
        "x-api-key": os.environ["ANTHROPIC_API_KEY"],
        "anthropic-version": "2023-06-01",  # required; omitting it is a 400
        "content-type": "application/json",
    }
    payload = {
        "model": os.environ.get("CLAUDE_MODEL", "claude-sonnet-4-5"),
        "max_tokens": 1024,
        "messages": [{"role": "user", "content": prompt}],
    }
    async with httpx.AsyncClient(timeout=60) as client:
        for attempt in range(1, max_attempts + 1):
            r = await client.post(ANTHROPIC_URL, headers=headers, json=payload)
            if r.status_code == 200:
                return r.json()["content"][0]["text"]
            try:
                body = r.json()
            except ValueError:  # corporate proxies love returning HTML error pages
                body = {"error": {"message": r.text[:300]}}
            verdict = classify(r.status_code, r.headers, body)
            req_id = r.headers.get("request-id", "n/a")
            if not verdict["retry"] or attempt == max_attempts:
                raise RuntimeError(f"{verdict['why']} (request-id {req_id})")
            # Full-jitter backoff, but never shorter than what the server asked for.
            sleep_for = max(random.uniform(0, min(30, 2**attempt)), verdict["wait"] or 0)
            log.info("attempt %d failed (%s), sleeping %.1fs", attempt, verdict["why"], sleep_for)
            await asyncio.sleep(sleep_for)
    raise RuntimeError("unreachable")


@mcp.tool()
async def summarize(text: str) -> str:
    """Summarize text in five bullets using Claude."""
    # Let RuntimeError propagate: FastMCP turns it into an isError tool result,
    # so the client sees our one-line message instead of a stack trace.
    return await call_claude(f"Summarize in five bullets:\n\n{text}")


if __name__ == "__main__":
    mcp.run()

A few notes on the design.

classify is written so it doesn't care which provider sent the error. Anthropic puts the error kind in error.type. Gemini puts it in error.status (for example RESOURCE_EXHAUSTED). OpenAI uses both type and code. The chained or picks whichever field exists. That's why the same function works when I point a variant of this server at OpenAI or Gemini, and I've done that twice.

The backoff takes the larger of the jittered value and the server's hint. Most retry snippets I see online do one or the other. If you ignore the hint you retry too early and get another 429. If you rely only on the hint you lose the jitter, so ten parallel tool calls all wake up in the same millisecond and hit the rate limit together.

The MCP piece is small on purpose. FastMCP catches exceptions raised by a tool and returns them as a tool result with isError: true. The model gets the message and can decide what to do next. In my testing it handles "insufficient_quota: add credits" well. It stops and tells the user, which is exactly what I want.

How the three big APIs report failure

These are the differences that actually changed my code. I left Mistral out of the table because I've only seen a handful of its error bodies and don't trust my sample.

The "out of money" row is the trap. All three providers report it differently, and two of them use the same status code as an ordinary rate limit.

What went wrong the first time

The first version of classify retried every 429. It started life in an OpenAI-backed variant, and one night that variant ran against a project whose prepaid balance had reached $0.00. With max_attempts=20 and no cap on the backoff, a batch job spent 47 minutes backing off politely from an account that was never going to answer. I was wrong to assume that a 429 always clears up on its own. I'd read the docs. I just didn't believe them until it cost me an evening.

The second failure was sillier. While debugging I added a print() call. stdio MCP servers use stdout for JSON-RPC, so that single line of debug output corrupted the stream, and Claude Code reported the server as disconnected with no further detail. Honestly this annoyed me for a day, because the server ran fine when I launched it by hand. Everything now goes to stderr through logging, and the comment at the top of the file is there so future me doesn't repeat it.

Third, there's a problem I haven't solved. Gemini reports a per-minute rate limit and an exhausted daily free-tier quota with the same RESOURCE_EXHAUSTED status, and both can include a retryDelay. My code retries both. For the daily case that's five pointless attempts. The QuotaFailure details include quota identifiers that look like they'd separate the two, but I haven't checked that those strings are stable across models, so I'm not matching on them yet. If you've shipped something that handles this cleanly, I'd like to see it.

The last one went in my favor by accident. Anthropic's low-credit error comes back as a 400, and my code never retries 400s, so it failed fast without any special handling. I didn't plan that. I'll take it.

FAQ

Q: Should I use the official Anthropic SDK instead of raw httpx?

A: For most apps, yes. It has built-in retries, and it raises typed exceptions such as RateLimitError and APIStatusError. I used raw HTTP here so you can see each header and body field. That makes it easier to port classify to providers whose SDKs behave differently.

Q: Why not retry 400 errors once, just in case?

A: A 400 means the request itself is wrong: a malformed payload, a missing header, too many tokens, or no credit. Sending the same bytes again gives you the same answer. Change the request first.

Q: Does the MCP client retry failed tool calls on its own?

A: Not reliably. Some agents will call the tool again if the error message suggests that might help, which is one more reason to make the message readable. Don't count on it, and keep the retry logic in the server, where you control it.

Q: What should the agent see when retries run out?

A: One line that includes the status, the error type, and a request ID. The ID matters more than people think. It's the first thing provider support asks for, and you can't get it back after the process exits.

Written with AI assistance and human review. Try the tool at aidevhub.io/ai-api-error-decoder.