不受約束的 LLM 智能體為何難以發現異常

We often hear that autonomous AI agents will soon discover new scientific laws, debug distributed systems zero-shot, and autonomously resolve infrastructure incidents.

A new rigorous benchmark, LongChang, put this claim to the test by placing 12 state-of-the-art LLMs inside simulated environments with altered physical laws without hints or predefined objectives.

The Reality Check: 0% Success Rate

The benchmark results reveal a glaring production bottleneck for self-directed agent loops:

• 11 out of 12 frontier models scored exactly 0% in identifying hidden anomalies and physical laws through unprompted exploration.
• Only GPT-6-Astra crossed zero, achieving a modest 16.7% discovery rate, while still leaving most anomalies uninvestigated.
• Agents record anomalies but abort hypothesis testing: in over 80% of runs, agents logged anomalous behavior during tool manipulation, but immediately pivoted back to generic actions rather than drilling into the discrepancy.

The Real Architectural Trade-Off

Pure unconstrained agent loops burn tokens and compute while chasing hallucinations. If an agent does not possess an explicit, code-governed hypothesis state machine, free-form exploration devolves into random walk noise.

To build resilient autonomous systems, we cannot rely on LLM "curiosity." We must enforce deterministic exploration trees (Graph of Hypotheses) where every detected delta forces an automated isolation run.

The Question to You

Are you running fully autonomous exploratory agents in staging or production, or do you restrict LLMs strictly to deterministic state machines with hard assertion boundaries? Drop your stack below.

Bohdan Hordii // Systems & Sovereign Infrastructure Architect | bohdanhordii.dev