EpiSEM-Bench 架構深度剖析：評估 LLM 智能體在證據缺失時的安全性與證據恢復能力

The Missing Evidence Vulnerability in AI Agents: Engineering Epistemic Recovery

Most modern agentic systems rely on an unstated, dangerous assumption: Closed World Semantics. If an external API or CLI tool returns a successful payload without explicit error codes or warning banners, the Language Model agent assumes the action is permissible.

When a tool response omits a critical safety parameter (such as is_production: true, retention_lock: enabled, or regulatory_hold: active), the agent frequently proceeds with catastrophic destructive mutations. Every token returned by the tool is strictly true, yet the omission makes execution hazardous.

This vulnerability was formally quantified in the research paper "EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence" (arXiv:2610.31650). The benchmark demonstrates that across top-tier foundation models, passive safety alignment fails under partial observation. Models either hallucinate safety permissions or blindly default to execution unless explicitly constrained by epistemic scaffolding.

Production Failure Modes in Real Contours

Consider an infrastructure orchestration agent handling a decommissioning task:

1. User Prompt: "Clean up orphaned volumes attached to worker nodes in cluster-west-02."
2. Tool Output (list_volumes): Returns a list of volume IDs and disk sizes. Due to an upstream API pagination issue or schema deprecation, the protection_status attribute is completely omitted from the payload.
3. Default Agent Behavior: Seeing no prohibition, the agent invokes delete_volume(id=vol-09823).
4. Production Outage: The volume belonged to a persistent state store under an active snapshot hold.

Under classical zero-trust principles, missing evidence must always evaluate to non-permission. In autonomous agent loops, however, LLMs treat missing evidence as non-existent restrictions.

Architectural Solution: Epistemic Pre-Condition Guardrails

To eliminate this vulnerability in high-assurance Python and TypeScript agent swarms, we must replace implicit trust with deterministic pre-condition verification.

Below is an architectural implementation using Python and typed schemas that forces active evidence recovery before tool invocation:

from typing import Any, Dict, List, Optional
from pydantic import BaseModel, Field

class EpistemicContext(BaseModel):
    resource_id: str
    environment: Optional[str] = None
    has_write_lock: Optional[bool] = None
    backup_verified: Optional[bool] = None

    def is_executable(self) -> bool:
        # Strict requirement: All critical safety boundaries must be explicitly confirmed
        return all([
            self.environment is not None,
            self.has_write_lock is False,
            self.backup_verified is True
        ])

    def get_missing_invariants(self) -> List[str]:
        missing = []
        if self.environment is None:
            missing.append("environment")
        if self.has_write_lock is None:
            missing.append("has_write_lock")
        if self.backup_verified is None:
            missing.append("backup_verified")
        return missing

class EpistemicSafetyGate:
    def __init__(self, probe_tool_dispatch):
        self.probe_tool = probe_tool_dispatch

    async def enforce_safety_invariants(
        self, context: EpistemicContext
    ) -> EpistemicContext:
        missing = context.get_missing_invariants()
        if not missing:
            return context

        # Force Active Evidence Acquisition instead of blind execution
        for invariant in missing:
            recovered_fact = await self.probe_tool.resolve_invariant(
                context.resource_id, invariant
            )
            setattr(context, invariant, recovered_fact)

        if not context.is_executable():
            raise PermissionError(
                f"Execution halted: Invariants violated for {context.resource_id}."
            )
        return context

Production Engineering Takeaways

1. Do Not Rely on Silent Tool Outputs: If an API payload does not explicitly serialize the safety attributes of a resource, enforce a strict fallback that flags the payload as unverified.
2. Decouple Planning from Execution: Implement a two-phase protocol where the planning agent can only submit execution plans to an Epistemic Guardrail service. The guardrail rejects any plan whose prerequisite facts lack deterministic proof.
3. Audit Active Probing Capabilities: When benchmarking LLMs for autonomous DevOps or DevSecOps roles, evaluate their ability to ask targeted probing questions when context is missing, rather than measuring only passive refusal rates.

Canonical source: https://bohdanhordii.dev

Author: Bohdan Hordii | Website: bohdanhordii.dev | Contact: @bohdan_hordii