週末建置紀錄：為免費方案變更記錄摘要示範預演四種端點情況

The shippable part of this weekend's side project was not the feature. It was a roughly 60-line rehearsal suite that rate-limits, hangs, and corrupts its own model endpoint on purpose, then asserts the demo reports degraded instead of throwing a 500.

The feature itself — a single page that turns a changelog into three support-friendly bullets — took an afternoon. The rest of the weekend went into making failure the normal case, because a demo that only works when the model endpoint is healthy is not a demo, it is a coin flip.

The build, and the cut that made it fit two days

The project is deliberately small: one HTML page, one Python process, one outbound call. A pasted changelog goes to a chat-completions endpoint, comes back as text, gets parsed into at most three bullets, and renders with a banner showing whether the answer came from the model or from the fallback.

The constraint that shaped everything: both the model access and the host are free options. Neither is something to build a latency budget or an availability assumption on. So the architecture question changed from "how do we keep the model up" to "what does the page show in the two seconds after the model does not answer".

The cut list, written before any code

Scope was cut on paper first, with a rule instead of a mood: anything that needs a database or a second network hop is out. The table below is the actual list, kept in the repo as CUTS.md so the next weekend does not relitigate it.

One attempt plus one fallback replaced retries. That decision is the reason the rehearsal suite below is short: with a single attempt, there is exactly one failure path to prove.

Artifact 1: a model endpoint that misbehaves on command

The stub speaks just enough of the chat-completions shape to be swapped in via an environment variable. It is stdlib only, so the suite runs on a clean machine with no install step.

# stub_model.py — a switchable stand-in for the model endpoint
import json, time
from http.server import BaseHTTPRequestHandler, HTTPServer

MODE = "ok"  # ok | rate_limited | hang | malformed

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        self.rfile.read(int(self.headers.get("Content-Length", 0)))
        if MODE == "rate_limited":
            self.send_response(429)
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            self.wfile.write(b'{"error":"rate_limited"}')
            return
        if MODE == "hang":
            time.sleep(30)  # deliberately longer than the client budget
        if MODE == "malformed":
            body = b'{"choices":[{"message":{"content":"plain prose, no bullets"}}]}'
        else:
            body = json.dumps({"choices": [{"message": {"content": "- fix login\n- bump deps\n- ship it"}}]}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass

def serve(port=8765):
    return HTTPServer(("127.0.0.1", port), Handler)

The four modes map to the four ways a free endpoint actually disappoints: it answers, it pushes back, it stalls, or it returns a shape the parser did not expect. No fifth mode was added, because unmodeled failures cannot be rehearsed anyway.

Artifact 2: one attempt, one budget, one honest fallback

The client has no retry loop. It has a timeout, a parser, and a return value that always has the same keys.

# summarizer.py — one attempt, one budget, one honest fallback
import json, os, urllib.request

BUDGET_SECONDS = 2.0

def summarize(changelog, base_url=None, timeout=BUDGET_SECONDS):
    base_url = base_url or os.environ["MODEL_BASE_URL"]
    req = urllib.request.Request(
        base_url.rstrip("/") + "/chat/completions",
        data=json.dumps({"messages": [{"role": "user", "content": changelog}]}).encode(),
        headers={
            "Content-Type": "application/json",
            "Authorization": "Bearer " + os.environ.get("MODEL_KEY", ""),
        },
    )
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            text = json.load(resp)["choices"][0]["message"]["content"]
        bullets = [l.lstrip("- ").strip() for l in text.splitlines() if l.strip().startswith("-")]
    except Exception as exc:  # timeout, 429, bad JSON, shape drift
        return {"ok": False, "reason": type(exc).__name__, "bullets": [], "degraded": True}
    if not bullets:
        return {"ok": False, "reason": "empty_or_unparsed", "bullets": [], "degraded": True}
    return {"ok": True, "bullets": bullets[:3], "degraded": False}

The broad except is intentional and narrow in scope: the only thing inside the try is the network call and the parse, so nothing else can be swallowed. Every exit path returns the same schema, which is what lets the UI render one banner instead of a stack trace.

Artifact 3: the rehearsal suite

# test_rehearsal.py
import threading, time, unittest
import stub_model, summarizer

class Rehearsal(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls.server = stub_model.serve()
        threading.Thread(target=cls.server.serve_forever, daemon=True).start()

    @classmethod
    def tearDownClass(cls):
        cls.server.shutdown()

    def run_mode(self, mode, budget=1.0):
        stub_model.MODE = mode
        result = summarizer.summarize("- fix login\n- bump deps",
                                       base_url="http://127.0.0.1:8765", timeout=budget)
        stub_model.MODE = "ok"
        return result

    def test_happy_path(self):
        r = self.run_mode("ok")
        self.assertTrue(r["ok"])
        self.assertEqual(len(r["bullets"]), 3)

    def test_rate_limited_degrades(self):
        r = self.run_mode("rate_limited")
        self.assertTrue(r["degraded"])
        self.assertFalse(r["ok"])

    def test_hang_hits_budget(self):
        started = time.monotonic()
        r = self.run_mode("hang", budget=0.5)
        self.assertLess(time.monotonic() - started, 1.5)
        self.assertTrue(r["degraded"])

    def test_malformed_payload_degrades(self):
        r = self.run_mode("malformed")
        self.assertTrue(r["degraded"])

if __name__ == "__main__":
    unittest.main()

Run it with python -m unittest -v test_rehearsal. The hang case is capped at 0.5 seconds by design, so the whole suite cannot exceed two seconds even on a slow laptop — that follows from the budgets in the code, not from a benchmark.

The suite is not a quality test of any model. It is a contract test of the demo: whatever the endpoint does, the page renders. Model output quality is a separate, unsolved problem that this weekend did not touch.

Where the free model access and the free server option fit

MonkeyCode is the reason this particular weekend was possible at all: it provides free model access and a free server option, both as described by the operator rather than measured here.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Those two facts changed the design in two concrete ways. Free model access meant the endpoint had to be treated as intermittent, which is why summarize() returns a degraded flag instead of retrying. The free server option meant the deployment target was one process with no database, which is why persistence landed on the cut list.

Nothing about the workflow depends on a specific provider. The stub is local, and MODEL_BASE_URL decides where the live call goes. Quotas, model names, and host specifications are deliberately absent from this log because they change; the project's own pages are the place to read the current terms.

A dated smoke log instead of a stale claim

Free tiers move. A sentence like "the endpoint works" has a shelf life measured in days, so the demo writes evidence instead of asserting it.

MODEL_BASE_URL=... MODEL_KEY=... python - <<'PY'
import json, os, time, summarizer
t0 = time.monotonic()
r = summarizer.summarize("- fix login\n- bump deps\n- ship it", timeout=8)
host = os.environ["MODEL_BASE_URL"].split("//")[1].split("/")[0]
print(json.dumps({"ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
                  "host": host,
                  "ms": round((time.monotonic() - t0) * 1000),
                  **r}))
PY

Each run appends one JSON line to smoke.log. The /healthz endpoint reads the last line and returns stale when it is more than a day old, and the page shows that state in the same banner used for fallback. A demo that reports its own staleness is more useful than one that pretends to be fresh.

What the demo skips, and who should not copy this

Skipped, deliberately and on the record:

• No auth, no multi-tenancy, no saved history.
• No streaming, no multi-turn, no prompt templates per customer.
• No retry storm; one attempt, one fallback, one log line.
• No answer-quality eval. The parser only checks shape, not usefulness.

This approach is a poor fit for anyone who needs an audit trail, per-user isolation, sub-second responses, or model output treated as authoritative. It is also the wrong base for a product with a guaranteed quota, because the entire design assumes the endpoint can disappear mid-request. For a weekend demo that has to survive being posted publicly, the trade is the right one.

The reusable part is small: a stub that fails on command, a client with one attempt and one budget, and a log with timestamps. Point the smoke script at whatever endpoint is in use this weekend, keep the log dated, and let the rehearsal suite — not optimism — decide whether the demo is ready.