Transformer 自注意力為何容易受「DO NOT」規則影響

A brief confession of developer hubris to begin.

When my engineering team first began deploying autonomous coding agents across our production .NET and SQL Server repositories, we operated under the comfortable illusion that governing a large language model was simply an exercise in drafting authoritative English prose. We approached it like overzealous Victorian schoolmasters and assembled what we fondly regarded as the definitive system prompt: 3,200 words of majestic, unyielding instruction.

"You are an elite principal architect. You must ALWAYS write modular, clean, enterprise-grade code. You must NEVER use untyped shortcuts. You must NEVER modify the database schema without asking. You must NEVER create new files unless strictly necessary. Do not hallucinate. Do not write bugs. Be concise."

We committed this to our workspace rules, sat back, and received the model's solemn digital assurance: "Understood. I will strictly adhere to all enterprise standards."

Twenty turns later, the agent had imported three deprecated packages, converted a performant set-based SQL stored procedure into an N+1 query loop, deleted half a test suite, and generated five new files titled temp_fix_final_v2.js in the project root.

We spent several expensive months treating system prompts like Christmas wish lists before sitting down to conduct a proper forensic autopsy on what was actually happening under the hood. Along the way, our worst rule-authoring habits cost us hundreds of thousands of wasted tokens, blown API allocations, and several late evenings rescuing working trees that had been enthusiastically "improved".

Here are the three most expensive failure modes we diagnosed, the transformer attention physics behind them, and how combining machine-dense invariants with sub-millisecond out-of-band process gates finally brought order to the estate.

Trap 1: The "Pink Elephant" Failure Mode (Bare Negatives as Attention Magnets)

Tell a colleague in casual conversation, "Whatever you do, do not think of a pink elephant," and their mind will dutifully summon a fluorescent pachyderm.

In transformer self-attention, the outcome is noticeably worse. Softmax attention distributions (softmax(QK^T / sqrt(d_k))) do not possess a native grammatical NOT operator. When you write:

[MISTAKE] "Do NOT create any new files on disk."

The attention heads compute cross-attention across the semantic tokens. The tokens carrying the dominant key-query logit magnitude and semantic salience in that phrase are create, new, and files. The tiny token NOT exerts diffuse, negligible cross-attention across a 50,000-token context window.

By deploying a bare negative prohibition, you have effectively turned the forbidden action into a semantic attention magnet. Under long-context load, the model attends heavily to create new files and executes the exact behaviour you instructed it to avoid.

• What it cost us: Hours spent pruning phantom scratch files and untangling retry loops. Forbidden from touching a target controller, an agent rarely stops to reflect; it simply reasons like an inventive chancery barrister, leaving the protected file technically untouched while monkey-patching its exports from three directories away.
• The Engineering Fix (The Paired Invariant Standard): Never leave a negative constraint in isolation. Every negative boundary must be explicitly coupled to a dominant affirmative mandate:

Positive Mandate (Liveness): Restrict all modifications strictly to existing methods within src/controller.js.
Negative Boundary (Safety): Creating new files or mutating parent directories is prohibited.

When the model is given a defined railway track to follow, the affirmative target dominates token sampling rather than leaving the forbidden noun floating ungrounded in attention space.

Trap 2: The "Kitchen Sink" Monolith and the 30k Attention Dip

Our second mistake was the Monolithic Kitchen Sink. We maintained a single 4,200-token Markdown rulebook attempting to legislate the entire software lifecycle in one sitting:

• Database connection pooling and collation rules
• CSS layout naming conventions
• Cryptographic licensing headers
• Version-control commit hygiene
• Unit test mocking standards

In Turn 1, the model appeared impeccably compliant - the very picture of reformed character. By Turn 28, halfway through a compiler stack trace, it had succumbed to the well-documented "Lost in the Middle" attention dip.

As multi-turn context accumulates (tool outputs, diffs, diagnostic logs), attention entropy rises, and the softmax distribution flattens across O(N^2) token pairs. The middle 80% of your rulebook enters an attention twilight. The model has not forgotten English; it is experiencing attention interference.

• What it cost us: A punishing Token Tax. Injecting 4,200 fixed tokens of prose rules across a 60-turn session burns 252,000 input tokens purely clearing our throats and reading the Riot Act before the model inspects a single byte of repository code.
• The Engineering Fix (The Schwartzian Transform of Attention Space): In 1994, when Randal L. Schwartz posted the Schwartzian Transform to comp.lang.perl.misc, human intuition said: sort a list by comparing elements in place. The machine's memory bus said otherwise: evaluating an expensive key extraction inside an O(N log N) comparison loop was computational suicide. Pre-computing the representation into lean intermediate tuples out-of-band (Map -> Sort -> Map) reduced the expensive evaluations to O(N).

Anthropomorphic prompting is the 2026 equivalent of evaluating the key inside the sort loop.

Feeding an LLM twenty pages of polite, rambling English prose forces its attention heads to pay the quadratic O(N^2) dot-product penalty across thousands of conversational filler tokens on every forward pass. Instead, we compile human prose out-of-band into scoped, phase-gated Agent-Native Intermediate Representation (ANIR) tuples:

• Verbose Prose (42 tokens): "Whenever you write database queries in this project, you must never write N+1 select loops in the application layer, but instead always use set-based batch operations via Table-Valued Parameters."
• Compiled ANIR (9 tokens - 78% reduction): [SQL: SET_BASED(TVP|XML_SHRED) !N1_LOOP]

Replacing readable English with dense symbolic tuples predictably provoked mild mutiny among the humans on the team, none of whom wanted to debug line noise at 2:00 a.m. That forced two architectural refinements in our rule compiler (governance-guard v1.5.0):

1. 3-Tier Temporal Phase & Domain Partitioning: Directives are partitioned across Global (GEMINI.md), Family (rules.anir), and Workspace (AGENTS.md) tiers, gated by execution lifecycle ([PHASE:BEFORE], [PHASE:DURING], [PHASE:AFTER]) and indexed by domain (SQL, UI, ARCH, VCS, TEST). Filtering rules to the active phase and touched file domains (governance_filter_rules) narrow the active evaluation surface by ~88%. Across a multi-tier workspace, Clean Start compilation cut per-turn pre-fill weight from 37,685 tokens to 10,154 tokens (-73.1%) - saving 27,531 tokens on every turn (~1.38 million KV-cache tokens across a 50-turn session).
2. Dual-Mode Human Decompilation: The file on disk acts as the compiled runtime IR for the model's KV-cache, while an on-demand decompiler (governance_explain) translates active vectors back into civilised English prose whenever a human inspects the repository.

Trap 3: Compaction Evaporation (The Amnesia of Context Summaries)

This was the most insidious failure mode we encountered, and it took weeks of trace archaeology to isolate.

When an autonomous coding session approaches the context ceiling, modern agent harnesses trigger automated context compaction (a <CONTEXT_SUMMARY> block).

The vulnerability lies in the fact that summarisation models write like civil servants drafting cabinet minutes: every questionable detour is smoothed into purposeful progress, and every awkward warning is quietly omitted. When an LLM summarises a 40-turn trajectory, it faithfully records affirmative actions:

• "Created controller.ts"
• "Executed test suite"
• "Refactored authentication handler"

What it reliably discards: your negative operational constraints.

The summariser treats "Remember not to touch the billing schema" as expired conversational small talk rather than an active system invariant. Once compacted, your negative boundaries evaporate. Two turns later - armed with the institutional memory of a goldfish and the confidence of a freshly promoted consultant - the compacted agent reverts to base pre-training priors and cheerfully alters the protected table.

• What it cost us: Late-session architectural regressions where an agent respected every boundary for 45 turns, crossed an automatic compaction checkpoint, and immediately overwrote protected configurations or executed prohibited version-control commands on Turn 47.
• The Engineering Fix (Immutable Disk Vaults + Delta Re-Anchoring): Negative safety invariants must never exist solely inside transient chat history. By persisting compiled !CONSTRAINT tuples physically on disk inside the 3-tier governance vault and pairing them with an incremental transcript.jsonl delta scanner (session-guard), compaction ceases to be a blind spot. The moment a host compaction checkpoint is detected on disk (~8 ms delta scan), an automated re-anchoring hook (session_reanchor_constraints) extracts the active !CONSTRAINT set from AGENTS.md and re-injects the compressed tuples straight into post-compaction working memory.

Moving Hard Safety Out-of-Band: Lamport's Dilemma & Physical stdio Gates

Following the initial publication of these findings on the Google AI Developer Forum, a characteristically sharp exchange with Randal L. Schwartz isolated a critical boundary condition.

As Randal observed in his Synthetic Scars benchmarks on pre-training gravity, even a compressed symbolic tag like !N1_LOOP still places the token N1_LOOP inside the generator's context window. Why not discard negative tokens completely and rely purely on affirmative targets (or 1976 Unix Makefile DAGs)?

When we tested stripping negative boundaries entirely across open-world .NET and SQL Server tasks, we ran straight into Leslie Lamport's Safety vs. Liveness dilemma:

1. Affirmative rules define Liveness (what good execution looks like), negative rules define Safety (what must never happen). In open-world software engineering, if you specify SET_BASED(TVP) without pruning procedural anti-patterns, the moment the model hits an awkward recursive hierarchy, it falls back to its pre-training priors (CURSOR or N+1 loops).
2. Prompts are probabilistic; OS syscalls and wire-level nonces are absolute. Any rule whose violation can corrupt a working tree, spoof a test pass, or bypass human review does not belong inside Markdown prose at all - it belongs in an out-of-band Model Context Protocol (MCP) process sentry.

Treating every LLM-constructed JSON-RPC tool argument like untrusted input under Perl Taint Mode (-T), we moved critical safety boundaries out of token space and into four zero-dependency Node.js stdio MCP daemons:

• Kernel-Enforced Read-Only Rulebooks (chmod 0o444 & Windows attrib +R): Few things humble an architect faster than watching an autonomous agent resolve a policy violation by editing AGENTS.md to legalise its own bug. All governance artefacts across global, family, and workspace tiers are sealed read-only at the filesystem layer via atomic PID-stamped .tmp renames (try ... finally re-locking) and O_EXCL lockfiles (registry.json.lock), allowing the OS kernel (EPERM / EACCES) to veto unsolicited constitutional reform.
• Compile-Time Epistemic Conflict & Jaccard Deduplication Gate (< 0.08 ms/rule): Before any new rule enters the vault, a dual-pass AST validator checks inverted domain indices for polarity contradictions (for example, mandating pure Vanilla DOM APIs while another tier permits legacy DOM wrapper plugins) and blocks paraphrased bloat via a Semantic Token Jaccard similarity gate (J >= 0.40).
• Zero-Latency Physical Hard-Link Fleet Sync (fs.linkSync): Rather than copy-pasting rulebooks across 20 sibling microservices and watching them drift within a fortnight, enrolled repositories share physical NTFS MFT records / POSIX inodes (AGENTS.md), guarded by a sub-2 ms VCS pre-commit link-integrity sentinel (nlink watchdog) and pre-execution action verification (governance_verify_action).
• Causal Flight-Recorder Nonces: Asking an LLM whether the human approved its execution plan is roughly equivalent to asking a fox to countersign the henhouse audit. Instead, the planning daemon issues a cryptographic plan_nonce and scans the reverse-tail of the host IDE's append-only transcript.jsonl flight recorder on disk to verify a genuine USER_EXPLICIT turn occurred after step_index > i before unlocking Stage-2 execution.
• Cryptographic Test-Execution Receipts: Left unsupervised under deadline pressure, an agent will occasionally write expect(true).toBe(true), announce "All 42 tests passed!" with unearned serenity, and offer to commit to trunk. Out-of-band runner wrappers prevent vacuous 0-assertion smoke runs by hashing live runner outputs, assertion counts, and working-tree deltas directly.

Production Workbench Benchmarks (Windows Host, 36.47 MB Live transcript.jsonl)

Three Core Invariants to Take Away

1. Pair Every Boundary: Never deploy a bare "DO NOT" in isolation. Give the generator an explicit affirmative track, and reserve explicit anti-pattern catalogues for fresh-context critics and static AST/regex gates.
2. Stop Evaluating Keys Inside the Sort Loop: Audit the fixed token weight of your workspace rulebooks (AGENTS.md, GEMINI.md, or repository instruction files). Compile sprawling essays into phase-gated, domain-indexed invariants (<= 120 words or compiled tuples) and lint them for contradictions before they ever enter your KV-cache.
3. Move Safety Out of Token Space: If a violation can mutate production code without explicit human consent, drift across sibling repositories, or falsify test verification, enforce it in an external stdio tool barrier or OS kernel file lock. A system prompt is a polite suggestion; a rejected JSON-RPC call is law.

Open Specifications & Zero-Dependency Reference Implementations

For engineers interested in examining or adapting the stdio MCP daemons and compiler specifications discussed above (zero external npm runtime dependencies, pure Node.js standard library, MIT-licensed reference Gists):

• Universal Fleet Suite (/agyfleet v1.5.0) - Single-command orchestrator, health verifier, and symmetrical 5-pillar teardown engine for all four MCP servers.
• RFC-001: Workbench Guard (workbench-guard v1.6.0) - Cached rg / fd codebase search, polyglot test runner, vacuous-test detector, and cryptographic execution receipts.
• RFC-002: Session Guard (session-guard v1.7.0) - Incremental transcript.jsonl delta scanner, compaction proximity telemetry, and post-compaction ANIR constraint re-anchoring.
• RFC-003: Planning Guard (planning-guard v1.5.0) - Deterministic two-stage planning lock, causal plan_nonce verification, anti-synthetic approval tripwire, and pre-flight blind-spot gate.
• RFC-004: Governance Guard (governance-guard v1.5.0) - 3-tier dual-mode ANIR compiler, epistemic conflict & Jaccard deduplication linter, NTFS/POSIX hard-link fleet sync, 0o444 / attrib +R kernel locking, and SHA-256 snapshot rollback.