部署就緒檢查清單如何為每道關卡設定有效期限

Last month a teammate asked me one question before a Friday deploy: "Is the summarizer ready?"

I answered from memory. That was the bug.

I had run the tests. I had done one manual canary. But I had no artifact that said when, or what changed since. My confidence was real. My evidence was a vibe.

So I built something small. Not a bigger checklist. A checklist where every pass expires.

The problem with green checkmarks

A green checkmark has no timestamp. It never says when it was earned.

So a pass from three weeks ago looks identical to a pass from this morning. The checklist stays green while the code moves underneath it. That is a silent failure mode, and silent failures are the expensive ones.

The fix is boring. Treat evidence like milk. Give it a date, give it a shelf life, and refuse to serve it after that.

What the dossier looks like

Five gates. One evidence file each. No prose, no opinions in the file — just a result and a timestamp.

readiness/
  gates.json
  readiness.py
  evidence/
    g1_input_contract.txt
    g2_canary.txt
    g3_cost_ceiling.txt
    g4_rollback_drill.txt
    g5_owner_handoff.txt

The config stays tiny on purpose.

{
  "ttl_days": 14,
  "gates": [
    { "id": "G1", "claim": "inputs are validated before the model sees them", "owner": "sam", "evidence": "evidence/g1_input_contract.txt" },
    { "id": "G2", "claim": "one canary task passed on the free tier", "owner": "sam", "evidence": "evidence/g2_canary.txt" },
    { "id": "G3", "claim": "worst-case cost per 1k calls is bounded", "owner": "sam", "evidence": "evidence/g3_cost_ceiling.txt" },
    { "id": "G4", "claim": "rollback was executed, not imagined", "owner": "sam", "evidence": "evidence/g4_rollback_drill.txt" },
    { "id": "G5", "claim": "someone other than me can turn it off", "owner": "sam", "evidence": "evidence/g5_owner_handoff.txt" }
  ]
}

Each evidence file is two lines. Nothing else.

RESULT: pass
RUN_AT: 2026-10-17T21:40:00Z

Step 1: Write the checker before you write the gates

I wrote the checker first. That forced me to decide what a valid pass actually is.

The whole thing is stdlib Python and about 65 lines.

#!/usr/bin/env python3
"""readiness.py - fail-closed evidence checker. Exit 0 only when every gate is green and fresh."""
import json, sys, pathlib, datetime as dt

ROOT = pathlib.Path(__file__).resolve().parent
STAMP = "%Y-%m-%dT%H:%M:%SZ"


def parse_stamp(line):
    if not line.startswith("RUN_AT: "):
        raise ValueError(f"missing RUN_AT line: {line!r}")
    return dt.datetime.strptime(line[8:].strip(), STAMP).replace(tzinfo=dt.timezone.utc)


def check_gate(gate, ttl_days, now):
    path = ROOT / gate["evidence"]
    if not path.exists():
        return "MISSING", f"{gate['id']} evidence file not found"
    lines = [l.strip() for l in path.read_text().splitlines() if l.strip()]
    if len(lines) < 2 or not lines[0].startswith("RESULT: "):
        return "MISSING", f"{gate['id']} needs RESULT then RUN_AT"
    result = lines[0][8:].strip().lower()
    if result != "pass":
        return "FAIL", f"{gate['id']} last run reported {result!r}"
    try:
        ran = parse_stamp(lines[1])
    except ValueError as exc:
        return "MISSING", f"{gate['id']} bad timestamp: {exc}"
    age = (now - ran).days
    if age > ttl_days:
        return "STALE", f"{gate['id']} evidence is {age}d old, ttl is {ttl_days}d"
    return "PASS", f"{gate['id']} ok ({age}d old, owner {gate['owner']})"


def main():
    cfg = json.loads((ROOT / "gates.json").read_text())
    now = dt.datetime.now(dt.timezone.utc)
    rows = [check_gate(g, cfg["ttl_days"], now) for g in cfg["gates"]]
    for status, msg in rows:
        print(f"{status:7} {msg}")
    bad = [r for r in rows if r[0] != "PASS"]
    print(f"\n{len(rows) - len(bad)}/{len(rows)} gates green")
    if bad:
        print("readiness: BLOCKED")
        return 1
    print("readiness: CLEAR")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Notice what it refuses to do. It does not run your tests. It does not judge quality. It only asks one question per gate: is there a recent, well-formed pass on disk?

Step 2: Make each gate produce its own evidence

The gate is not the check. The gate is the file the check leaves behind.

# G1: input contract
python -m pytest tests/test_input_contract.py -q \
  && printf 'RESULT: pass\nRUN_AT: %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  > readiness/evidence/g1_input_contract.txt

Repeat that shape for G4. Run the rollback for real, then record it.

# G4: rollback drill - flip the flag, confirm the fallback path, then restore
./scripts/flag.sh off && curl -s localhost:8080/health | grep -q degraded \
  && ./scripts/flag.sh on \
  && printf 'RESULT: pass\nRUN_AT: %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  > readiness/evidence/g4_rollback_drill.txt

For G2 I use one tiny canary task against free hosted model access instead of a paid endpoint. That keeps the gate cheap enough to re-run weekly, which is the whole point.

I have been doing this with MonkeyCode's free model access. I also run readiness.py on the free server option, so the check does not depend on anyone's laptop staying awake.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Step 3: Test the checker with a broken dossier

A checker you never saw fail is a checker you cannot trust.

So I keep a fixture with one stale file. It passes its own check and still blocks the deploy.

$ python readiness.py
PASS    G1 ok (2d old, owner sam)
PASS    G2 ok (2d old, owner sam)
STALE   G3 evidence is 41d old, ttl is 14d
PASS    G4 ok (3d old, owner sam)
MISSING G5 evidence file not found

3/5 gates green
readiness: BLOCKED
$ echo $?
1

That STALE line is the one I care about most. Nobody lied. The evidence was simply too old to mean anything.

The timebox that keeps this honest

A checklist that takes two days to satisfy gets deleted. Mine has a hard budget.

Total worst case is under three hours. If the feature is not worth three hours, it is not worth a Friday deploy.

Abandon and rollback criteria

Write these down before you need them. Mine are three lines.

1. Any gate stays red after two remediation attempts. Park the feature.
2. G3 exceeds the declared ceiling twice in a row. Cut scope, not the gate.
3. G5 has no secondary owner. Do not ship, no exceptions.

Rollback is one command and one file restore. If it needs a meeting, the gate failed.

Who should not use this

This is not for a weekend prototype with no users. It is also the wrong tool if you have no deploy path yet, or if nobody owns the feature.

And it will not save a team that treats evidence as paperwork. The script only checks freshness. It cannot check whether the canary was meaningful.

The thing that actually changed

My teammate stopped asking me if it was ready. She runs the checker and reads five lines.

That is the win. Not more confidence. Less of mine required.

If you try this, my one real question is about G5: who is the second person who can turn your feature off, and have they actually done it once? That field is where most small-team dossiers break, and I would like to know how yours holds up.

You can reproduce the whole thing in an afternoon. Free model access covers the canary, and the free server option covers the checker run — no card, no CI runner.