Stateless Language Agents 研究：無狀態智能體在長期自動化研究任務的最高預算測試中表現最佳

When a long agent run stalls, the reflex is to give it more. More tokens, longer sessions, bigger context windows. The assumption underneath is that experience compounds, so an agent that remembers everything it has tried should keep getting smarter as the run goes on.

A new paper, Stateless Language Agents: Scaling Long-Horizon Automated Research (Zhang et al., from Stanford, CMU, UW, SambaNova and Berkeley), ran that assumption at budgets up to one billion tokens. For the automated research systems they tested, it does not hold. And the fix they demonstrate is close to the opposite of the reflex: the strongest system in their comparison is built from agents that retain no conversation at all between invocations.

What actually happened in the long runs

The paper studies AutoResearch systems: agents that iteratively propose, implement, and evaluate ideas for open-ended problems like kernel optimization and cross-language transpilation. These are good stress tests because the objective is measurable and the runs are long.

At long horizons, three failure modes showed up in their traces, and none of them look like the model getting dumber:

1. Replaying a growing history. Every new step drags the full conversation along. Tokens get spent re-reading old attempts instead of trying new ones.
2. Duplicating work. In one FrontierSWE run, two parallel agents implemented the same feature from the same parent candidate and got the same score, then overlapped again on the next feature, even though one agent had posted a note claiming it.
3. Quietly stopping. In one GPU kernel run, the last real improvement arrived at 231M tokens. The run kept consuming tokens up to 1B anyway. Of the final 1,500 sessions, 98% made no tool calls at all. The agents had decided their solutions were optimal, and every restart resumed the same conversation that had reached that conclusion.

That last one is worth sitting with. The system was not out of budget. But in that run, no further improvement ever arrived, because each restart brought back the same context that had already decided to stop.

State versus context

The paper's core move is separating two things that many agent frameworks glue together:

• State: what persists across invocations. Candidate solutions, measured outcomes, what has been tried.
• Context: what the agent sees within a single invocation.

Their principle: stateful search with stateless agents. No agent carries its conversation across invocations. The harness owns the research state, and for every invocation it reconstructs a fresh, role-specific context from that state. The system retains selected research state in the harness: candidates and their measured outcomes, not full transcripts. What each one sees within a run becomes an explicit design decision instead of an accident of accumulated history.

The implementation has two roles. A stateless Advisor reads harness-summarized evidence across all search directions and turns it into concrete experiment assignments. Parallel Workers each see only their own assignment, their current candidate, and local feedback. Statelessness is enforced, not prompted: Workers run as unprivileged processes under Linux Landlock filesystem restrictions, the harness clears their saved sessions between trials, and candidates are evaluated outside the Workers' reach with protected evaluator code.

Two design details matter more than they look. The harness keeps each Worker's local improvements even when that candidate trails the global best, so alternative approaches get time to mature instead of being abandoned early. And the Advisor is itself stateless, so it re-derives its plan from current evidence each epoch rather than defending its own earlier reasoning.

The numbers

The evaluation compares SLA against EvoX, CORAL, and SwarmResearch on four FrontierSWE software engineering tasks, Anthropic's VLIW SIMD kernel optimization, three NVIDIA SOL-ExecBench GPU kernel tasks, and a cryptographic algorithm-design task from FrontierCS, at budgets up to 1B cumulative tokens. Kernel quality is measured in execution cycles, where fewer cycles means a faster kernel.

Two measurements matter, and they are separate questions: who ends ahead at the full budget, and how much search each method needed to get there.

• On the first: SLA achieved the best full-budget result on every evaluated task. On the Anthropic kernel task with Codex, all three SLA runs beat all nine baseline runs at 1B tokens. The worst SLA run finished ahead of the best baseline run (1,122 vs 1,191 cycles).
• On the second: SLA reached the strongest baseline's final kernel performance using 67.9M tokens versus the baseline's 986.3M, a 93.1% reduction (84% with Claude Code). On FrontierSWE it improved final scores by 4.95 points on average over the strongest per-task baseline.

The most interesting result is about evaluation, not architecture. On all four FrontierSWE tasks, SLA was losing at 25% of the token budget and winning at 100%. A short evaluation would have ranked it below the baselines on every one of those tasks. The ablations show the same pattern in miniature: removing Worker context isolation made the kernel slower by 5 to 11 cycles over a 100M-token continuation from a shared checkpoint, but by 224 cycles over a full 1B-token run (1,336 vs 1,112). The cost of that design choice compounds with the horizon: in this experiment, the shorter continuation substantially understated the penalty.

Explicit coordination also turned out to be nearly free in these runs. The Advisor consumes 0.24 to 0.51% of total tokens and at most 2.3% of model cost. SwarmResearch's long-running Shepherd coordinator burns 8.4 to 10.7% of tokens, largely on re-reading its own conversation.

What this suggests if you build agents

A few implications look like they transfer beyond research automation:

Stop treating the transcript as state. If an agent loop persists by appending to a conversation, its state management is "whatever happened to get said." The paper's alternative is to put durable state (candidates, results, decisions) in a structured store and build each step's context deliberately from it.

Rebuild the context when restarting a stalled run. In the stalled runs the paper describes, resuming the same session reproduced the stall. That is evidence for restarting with a context rebuilt from evidence, not from the conversation that gave up.

Parallel agents need assignments, not just shared memory. In their traces, "everyone reads everyone's notes" still produced duplicated features. Explicit assignments were one of the ablated components, and removing them measurably hurt.

Score your evals at multiple budgets. In this study, a method that trailed early dominated late, and the reverse. A single checkpoint ranking told you little about design choices whose costs compound over the run.

Honest limits

The paper is careful about its own scope, and it is worth keeping that care. Every task has an executable evaluator, so this says nothing about goals that are vague or slow to score. Most configurations ran only once, since a single long run costs hundreds to thousands of dollars, so treat the non-kernel numbers as directional rather than settled. And this is a design for autonomous search loops, not conversational assistants, where continuity of the dialogue is the product rather than the enemy. The lesson for interactive agents is narrower: keep the operational state of long tasks out of the chat history. Finally, this is an arXiv preprint; treat the results as early evidence.

The takeaway

In the long-horizon search systems tested here, the bottleneck was not how much the model could remember. It was that the conversation was the wrong place to keep what mattered. Moving state into the harness, rebuilding context fresh each step, and letting one cheap coordinator decide what gets tried next produced stronger results at a fraction of the tokens. Whether that generalizes past automatically scored search is an open question, but within these experiments, the agents that scaled were the ones allowed to forget.

Sources: Stateless Language Agents: Scaling Long-Horizon Automated Research (arXiv:2610.07625, October 2026), code