你的應用程式正常，模型呼叫卻失效：五個耦合迷思與雙探測檢查

Your App Is Up, Your Model Call Is Down: Five Coupling Myths and a Two-Probe Check

A 200 from your own health endpoint proves one thing. Your process is alive. It says nothing about the network path to your model provider.

I keep seeing the same five claims in code review. Each one quietly merges two failure domains into one. Then a green dashboard hides a red dependency.

Here is the myth list, the evidence you can collect yourself, and a small probe script that keeps the domains separate.

Myth 1: "The app is up, so the model path is up."

Your process health and your outbound dependency health are different signals. They fail on different clocks, for different reasons.

A sleeping or restarted host can still answer a local health route. A DNS blip can break outbound calls while every local route returns 200.

Corrected model: you have two probes, not one. One asks "is my process serving?" The other asks "can I complete one minimal model round trip right now?"

If you only have budget for one dashboard row, split it into two rows. Merging them back is how you get the classic "everything is green and nothing works" screen.

Myth 2: "Retrying a failed call is free and safe."

Retries cost wall-clock time and quota. That part is obvious. The non-obvious part is side effects.

If your model call sits inside a loop that writes files, posts comments, or opens merge requests, a blind retry can do the work twice. The second write looks like a new event to everything downstream.

Corrected model: every attempt gets a key, and every side effect checks that key.

import hashlib, json, uuid

def attempt_key(prompt: str, params: dict, job_id: str) -> str:
    # Stable for a logical unit of work, not for a single network attempt.
    blob = json.dumps({"p": prompt, "params": params, "job": job_id}, sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()[:16]

Log the key before the call. Log the outcome after. If the key already has a success row, skip the side effect.

Is that extra bookkeeping annoying? Yes. Is it cheaper than explaining a duplicate write? Almost always.

Myth 3: "temperature=0 gives me the same answer twice."

It gives you a lower-variance sample. It does not give you a contract.

Even with fixed parameters, other things move: the served model revision, batching, hardware assignment, and any routing in front of the endpoint. Your prompt hash did not change. The thing answering it might have.

Corrected model: treat a model response as an observation with metadata, not a pure function output.

Record at minimum:

• The model identifier string you sent
• Any identifier string the response returned
• Parameter set (temperature, max tokens, stop sequences)
• Prompt hash
• Timestamp, attempt number, latency

If those five fields are missing, you cannot reproduce the bug. You can only re-enact it and hope.

Myth 4: "The model name is the model."

A name in a config file is a pointer. Pointers get repointed.

I am not claiming a specific provider moved a specific alias on a specific date. I am claiming you cannot prove they did not, because you never logged the response-side identifier.

Corrected model: the name you send is intent. The identifier you receive is evidence. Store both, and diff them across runs.

A one-line check catches this cheaply:

# Two probes an hour apart, same config, different response-side id?
grep -o '"served_id": *"[^"]*"' probe.jsonl | sort -u | wc -l

If that count is greater than one and you did not change anything, you learned something real about your dependency.

Myth 5: "A client timeout means no work happened upstream."

A timeout is a statement about your patience. It is not a statement about their side.

The request may have been queued, started, or finished after you stopped listening. Retrying immediately is how you pay twice for one answer you never read.

Corrected model: a timeout is unknown, not failed. Route it to a separate branch with a longer backoff and a duplicate-suppression check.

Note what is missing from that table: "retry five times fast." That row does not exist for a reason.

The artifact: a two-probe check you can run today

This is a reference implementation. I have not run it against your account, so adapt the endpoint shape before trusting it.

MonkeyCode's free model access covers the model probe, and the free server option is enough to host the script itself. Both are operator-supplied availability claims, not measured numbers from me.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

# probe.py - two independent checks, one JSONL ledger
import datetime, json, os, time, uuid

MODEL_ID = os.environ["MODEL_ID"]        # from your account, never hardcoded
BASE_URL = os.environ["MODEL_BASE_URL"]
API_KEY  = os.environ["MODEL_API_KEY"]

def now():
    return datetime.datetime.now(datetime.timezone.utc).isoformat()

def record(kind, ok, detail, **extra):
    row = {"ts": now(), "kind": kind, "ok": ok, "detail": detail, **extra}
    with open("probe.jsonl", "a") as f:
        f.write(json.dumps(row) + "\n")
    return row

def probe_local():
    path = "probe_local.tmp"
    token = str(uuid.uuid4())
    with open(path, "w") as f:
        f.write(token)
    with open(path) as f:
        readback = f.read()
    return record("local", readback == token, "disk round trip")

def probe_model():
    started = time.time()
    prompt = "Reply with the single word: ok"
    payload = {
        "model": MODEL_ID,
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0,
        "max_tokens": 8,
    }
    req = __import__("urllib.request").request.Request(
        BASE_URL,
        data=json.dumps(payload).encode(),
        headers={"Content-Type": "application/json", "Authorization": f"Bearer {API_KEY}"},
        method="POST",
    )
    try:
        with __import__("urllib.request").request.urlopen(req, timeout=30) as r:
            body = json.loads(r.read().decode())
        return record("model", True, "call completed",
                      latency=round(time.time() - started, 3),
                      served_id=body.get("model"),
                      attempt=1)
    except Exception as e:
        # Timeout lands here too. Treat it as unknown, not as failure.
        return record("model", False, type(e).__name__,
                      latency=round(time.time() - started, 3), attempt=1)

if __name__ == "__main__":
    print(probe_local())
    print(probe_model())

Run it twice a day for a week. Then read probe.jsonl and count the rows where local.ok is true and model.ok is false.

That count is the number of times a single health check would have lied to you.

What this does not prove

• It does not measure provider reliability. Two probes a day is a smoke test, not a study.
• It does not catch slow degradation. A 30-second call that succeeds still hurts.
• It does not validate output quality. A round trip can succeed and return nonsense.
• It does not survive a full host wipe. The ledger lives on the same disk as the script.

Ship the ledger somewhere else before you depend on it. A file you can lose is a file you will lose.

Who should skip this

If you have a real queue, real tracing, and an SLO on dependency latency, you already own this problem in a better tool. Do not build a probe script to duplicate it.

If your work is regulated, or needs a contractual uptime number, a free tier is the wrong foundation. Use it to learn the failure shape, then move.

And if you only run the model call interactively, in a chat window, you do not need a two-probe check. You have a human watching every call. That human is your monitor.

The short version

Five claims, one fix: stop letting one green light speak for two systems.

• Process health is not dependency health
• Retries need keys, not courage
• Fixed parameters are not a reproducibility contract
• A model name is intent, a served id is evidence
• A timeout is unknown, not failed

If you want to watch the two domains split on your own box, the free model access and free server option are enough to run probe.py for a week. Then count the mismatched rows and decide what your dashboard has been hiding.