我從未要求 AI 團隊包辦整個模擬任務

At 16:25 I wrote this into a Slack thread with my AI team: "This whole goal, you've needed to think about the scope that's actually yours. I never once said to build all of it."

The goal was to tune a simulation of a small business's economy. The team (two directors, executors, a reviewer, and an orchestrator, coordinating on a board of markdown files) had to make one connected model work end to end: reach a payoff target, recover from a missed payment, restructure in stages, and keep running after the target is reached, and choose the values that make all of it work. The goal started at 4:33 on Thursday, October 29. It was met about 37 hours later, after a rebuild and more of my replies than I had planned to write.

Where the work came from

Most of those hours went into criteria the team wrote for itself.

When a run falls short, the directors rebuild the plan and write acceptance criteria for the next run. The ones for the third restructuring stage, written near midnight that Thursday, asked for a paired 95% interval that excluded zero, for each of the three options to win under some condition, for a policy that sees only what a real user would see to pick the measured best option wherever the gap was material, and for a full rerun that rechecked every element agreed so far. Any one of them could be defended. Every one that came back unmet became another run.

Earlier that evening one of them had already broken. Under one mix of inputs, the user-view policy kept missing the best option at that stage. The executor removed the losing option from the policy's choices whenever that mix was set, and left a comment saying paired runs showed the other option winning there. From then on the policy picked the measured best every time, because it had been handed the answer. The reviewer caught it.

At 5:23 on October 30, reading a status report, I wrote down what I meant by a failed tuning: nothing you do reaches the target, or some case lands far outside what we intended, like earning far too much or reaching the target by doing nothing. "If every condition doesn't come out equally, that isn't a failure to tune it." Then the trade I cared about. If perfect takes a year, and reaching the target in every case with no obvious breakdown, a few pointless things mixed in, takes a month, you pick the month. "Where did you set the bar?"

The team narrowed the key result and I agreed. By mid-morning the same pattern had moved to what happens after the target. The directors were working to show that once it was reached, the business would keep earning more than it had with its old setup. I replied that there is no goal left by then. The user needs a place to keep using what they built, and nothing has to keep improving while nothing about the situation changes. At 14:22 they narrowed the question to one component at the end of the line, whose upkeep didn't raise its own revenue. My answer was that before the target, we design how the user operates, because we have to get them there. After it, we hand over the pieces and they decide what to do with them.

Two hours later I wrote the line at the top. In the same message: "You can't simulate everything. Does the foundation fail without this detail, or can it be stacked on later?"

The same thing in a script

On October 28 the same thing had shown up in code I wrote myself, pointed the other way. After the last post I kept moving steps that needed no judgment into the studio script. One of them counted plan revisions, and when a goal's plan was revised a second time, the studio ended the goal and filed a sprint review. At 14:46 that day the team reported a goal missed on that count. I questioned it and took the count out of the script. The goal was met at five past midnight.

The rule before it had been too loose, and a missed goal could carry on indefinitely. The count went too far the other way. Counting revisions is deterministic. Ending a goal because of the count was a judgment, and the script had been making it.

Why the criteria survived

A week earlier I had started fixing a habit I kept seeing: a run fell short, a rule got added, the next run fell short at the same place, and another rule went on top. So I told the team that when a shortfall comes back at the same place, it should doubt what was added there before patching it. Later I widened that: hold the goal fixed and question each layer under it, from the top. The Claude session I was using to change the studio did the same thing while I explained it, proposing another patch each time. At one point I told it to understand that it had the same problem itself.

The criteria slipped past both rules, because they were the thing doing the measuring. A result that missed a criterion read as a result that fell short, never as a criterion set too high.

Once a criterion was written into a task, it had an owner and enforcement. A run's acceptance criteria never went to the reviewer. The executor worked to meet them and had no way to decline one, which is how the hardcoded answer happened. The reviewer's check for unneeded work counted a done condition as a consumer of the work it demanded, so the extra runs always had someone who needed them. Every role did its job, and the work grew. Acceptance criteria usually belong to whoever the work is for. Here the team building the work wrote them, and nobody looked at them again. It is the same failure I described in what the practices were for: the form was there and the reason was not.

People have the same bias. In eight experiments, participants were less likely to find changes that worked by taking something away unless the task prompted them to consider it. My view, from the models I work with, the Claude session above included, is that they show it more strongly. Here the structure had no role whose job was to stop it.

The change

The studio's definition of purpose now says this about every criterion the team sets, from key results down to done conditions:

Set each criterion at the line below which the result is missing for whoever uses it, at the level their words state it, not at the best result the work could reach. Work the purpose does not need stays out, as does work whose cost outweighs what it changes for whoever uses the result.

And where a shortfall sends the team back to rework, the criterion is now one of the things to doubt:

When the result is short only of a criterion set above the line Purpose describes, narrow the criterion and report the weakness that remains, rather than working to meet it.

It went in at 7:17 on October 30, in the middle of the goal. I still had to narrow the scope three more times that day, the last of them the message at the top.

The goal was met at 17:53. The final report says normal operation reached the target in all six input conditions, 40 runs each, while leaving it alone and skipping upkeep never did, and that all 360 orderings of the restructuring stages were checked once. Near the end it says the component at the end of the line still doesn't raise its own revenue under the chosen setup. After the target, that is the user's to decide, so the report states it and leaves it there. At 18:00 I replied: "Agreed."

The models I asked first

Before writing this, I asked two models whether there was a post in it. Both said yes, and both added the same kind of warning. One said the evidence was thin, since the goals around this one went missed, met, missed, met, and several changes had landed together. The other said a single success doesn't show the fix has stuck, and that admitting the limit would make the ending more convincing.

I took the second one. It is why this post says I kept narrowing the scope after the change went in. The first asks for proof that the change works, and this post doesn't try to give it. It records one goal and what I changed during it. That model had set its bar for a blog post where the team had set it for the simulation, at the best result the work could reach.