智能體系統的正確性不是可事後補上的功能

You cannot add correctness to a system after the system is built. You either build the system so a claim needs evidence before the claim exists, or you measure, after the fact, how often the claims were wrong. Most of the record below did the second and called the result progress.

Look at the record in order.

November 2025. Anthropic publishes a guide to harnesses for long-running coding agents. The failures the authors describe include an agent seeing partial progress and declaring the job done, and an agent marking a feature complete without proper testing. The fix: ask the model to self-verify.

February 2026. OpenAI describes building a product with zero hand-written code. Human review of pull requests is optional. Merge gates are kept to a minimum.

April 2026. A UC Berkeley team audits 13 agent benchmarks, rates every one critical risk and scores perfectly on some without solving a single task.

July 2026. HANDBOOK.md tests models on following a company handbook. The best configuration passes 36.2 percent of trials, and failed runs end with a confident claim of compliance.

September 2026. Google Cloud AI Research publishes RRSI and warns automated harness improvement overfits:

"such recursive evolution may overfit by memorizing the training tasks"

October 2026. The same group publishes Harness², a method for improving agents on tasks with no ground truth, and writes in the limitations:

"They do not guarantee correctness or establish that an adopted change will transfer to a different task."

Read the sequence as one argument. Every step makes agents more capable or more autonomous. Every step leaves the same question open: is the answer right. Each paper and post states the question honestly, then hands the question to a judge, a benchmark or the user.

Now the same months, from our side, out of the git history.

18 January 2026, first commit. The design document's first non-goal: never modify or retrain the model. All improvement comes from the structure around the model.

1 March 2026, a design for learning from our own runs. The rationale says, in plain words:

"Fully autonomous self-modification is unsafe and hard to trust."

28 March 2026, the first evidence and completion gate. A task closes only with evidence on record.

29 May 2026, the final authorization gate. A model gets a vote on the final answer, and rules counting the record get the last word.

1 July 2026, the specifications. Every rule written down, most of them carrying the production session behind the rule.

25 July 2026, the change gate. A code change ships only with the specification updated in the same commit.

None of those is a feature you switch on. Each one removes a way for the system to claim something unsupported.

I owe you the other half, because this series names our own failures too. The March learning design included an evaluator for proposed improvements. Reading the code this month, I found the evaluator never ran the proposed change. The evaluator scored the health of the last 24 hours of ordinary runs and approved or rejected on the result. A pass meant the baseline was healthy. A pass said nothing about the change. Google's Harness² does the part we skipped: run the change, compare against the original, then decide. We are fixing our evaluator, and the fix gets a post when the record exists.

The point stands, and the failure proves the point. Our evaluator was a bolted-on measurement, and a bolted-on measurement looked fine for seven months while measuring the wrong thing. The gates on the final answer were built in, and the gates caught the failures Act II describes.

Act II starts with those failures. Ours first.