在消費級硬件上運行 LLM——第 7 部：策略師 Sage：雲端判斷、本地執行

Part 7 of a series. Previously: Part 6 — The Orchestration Ceiling.

Part 6 made the team's execution reliable: by moving the control loop out of the model and into deterministic code, the orchestrator could carry out a plan without dropping steps. But reliable execution answers only half of the problem. A plan carried out faithfully is of little use once it meets a wall the plan did not anticipate — and for that the team needed a capability that execution reliability could not supply: judgment. Judgment was the one thing served poorly by any local model, and it is the single part of the system that runs in the cloud. This entry explains why, and what it bought.

The stall-versus-improvise problem

Earlier entries established Gatsby, the orchestrator, as "the honest operator": reliable, never fabricating a result, never improvising wildly. That discipline is exactly what an orchestrator needs, but it created a tension at the team's edges. When the team hit a genuine blocker — a download that failed, a dependency that was absent — an honest, non-improvising operator has only two available moves, and both are wrong. It can stall: report the obstacle and stop. Or it can improvise: depart from the plan, and risk exactly the kind of unfounded, made-up progress the honesty rules existed to prevent.

What was missing was a third option — adaptation: a way to rethink the approach without either giving up or inventing. The problem was that reliability and creative rethinking are, to some extent, opposing traits, and one agent had been asked to supply both.

Separating judgment from execution

The resolution was to stop asking a single agent to be both dependable and inventive. Execution — the honest, faithful carrying-out of a plan — stays with Gatsby. Judgment — the creative rethinking that a wall calls for — moves to a new role: Sage, the strategist. Gatsby no longer improvises; when he is blocked, he consults Sage, and Sage does the adapting. Judgment lives in Sage; honest execution lives in Gatsby. The escalation was made explicit with a single rule added to Gatsby's instructions — when blocked, consult Sage — so that hitting a wall triggers a consultation rather than a stall or a guess.

Why this role — and only this role — is in the cloud

The operating principle set out earlier in the series was to run locally whatever reasonably can be, and to reserve the cloud for the few workloads that genuinely warrant a frontier model. Strategy is that workload, for two reasons.

First, it is invoked rarely — only when the team is actually stuck — so its cost is negligible. A role called a handful of times across a long run is precisely the kind of workload a free cloud tier serves without strain, and it consumes no local resources at all. Second, it is the one role whose quality turns on depth of reasoning rather than on throughput, and depth is where a frontier model sits in a different class.

That difference is measurable. Asked what makes a strategy good, the cloud model spent 413 internal "thinking" tokens before producing a 24-token answer — it reasoned for roughly seventeen times the length of what it ultimately said. That ratio is the entire point of the role: a strategist should think far more than it speaks. The local models used elsewhere in this system do not reason internally in this way; the capability is specific to the frontier model, and it is exactly what the role requires.

The free-tier reality

The specifics mattered, because the free tier had narrowed by mid-2026. Gemini's Pro models left it on 1 April 2026, leaving only the Flash and Flash-Lite tiers free. The role therefore landed on gemini-3-flash-preview: free, with a one-million-token context window — large enough that the context limits that shaped the local models are simply irrelevant here — and configurable "thinking" levels. A frontier-class strategist, consulted occasionally, at no cost.

Sage in practice

The value showed on a real blocker. The team was building a game and could not download the graphics it wanted; the network was blocked, and the workers treated this as a hard stop. Sage reframed it. The team, it observed, was conflating graphics with external downloads — and the two are not the same thing. It recommended procedural synthesis: generating the graphics in code rather than fetching them, with concrete next steps for each member of the team.

The detail that marked this as judgment rather than a lucky guess was anticipatory. Sage also directed the team to audit the project for other hidden external dependencies — before they hit the same wall somewhere else. That is the difference a reasoning model makes to the role: it did not merely solve the blocker in front of it, but recognised the class of problem behind it and moved to pre-empt the next instance.

A second case showed the same judgment turned the other way — toward stopping. Tasked with installing a set of system packages, Nova, the sysadmin, hit a wall that no amount of reframing could get past: the installation required a sudo password, which the agents did not have and should not have. Here the escalation path proved its worth in the opposite direction. Rather than inventing a workaround or declaring false success, the team routed the blocker up through Sage, and the judgment that came back was to surface it to the human. Credentials are exactly the kind of thing an autonomous system should stop and ask for, and the design held end to end: no fabrication, no stalling in place, just an honest hand-off to the one party who could supply what was missing.

What follows

With judgment and execution separated, the team could at last both stay honest and adapt. But a strategist that reframes a problem well does nothing to guarantee that the work it unblocks is actually verified — and verification turned out to be the team's weakest link. The next entry is about quality assurance, and about why "it ran" and "it works" are not the same claim.