如何設計不產生幻覺的生產級多智能體系統,以及何時不該使用 LLM
How to Architect a Production Multi-Agent System Without Hallucinations (And When Not to Use an LLM)
生產級多智能體系統應以明確架構控制故障,而非只靠加長提示詞;對確定性任務使用規則路由,僅將模糊情況交給 LLM。文章提出以有界狀態圖、結構化驗證、混合向量與圖檢索,以及高風險操作的人手審批,降低錯誤累積與無限循環。攻略亦列出上線前的權限、提示詞注入過濾、追蹤記錄和故障降級檢查。
Most AI agent demos fail the moment they touch real business data. Not because the underlying models aren't capable, but because engineers treat probabilistic LLMs like deterministic microservices.
At WTFearB, our core engineering philosophy is simple: Make What If Work. When founders and operations teams ask, "What if the agent hallucinates a refund?" or "What if latency spikes under load?", the answer isn't a longer system prompt—it's better software architecture.
Here is the exact blueprint we use when engineering custom AI workflow automation and multi-agent systems for production environments.
Why Do Multi-Agent Systems Fail in Production?
Multi-agent systems fail in production primarily due to unbounded state loops, unvalidated tool outputs, and routing simple deterministic logic through expensive LLMs. Without strict state machines and schema enforcement, small reasoning errors compound across every agent handoff.
| Failure Mode | Demo Approach (Fragile) | Production Architecture (Resilient) |
|---|---|---|
| Intent Routing | Asking an LLM to decide every next step | Deterministic classifier / semantic router first; LLM only for ambiguity |
| Tool Execution | Raw JSON parsing from free-form text | Strict Pydantic / JSON Schema validation with automatic retry budgets |
| Memory & Context | Stuffing entire chat histories into the prompt | Scoped state graphs + hybrid vector/graph retrieval (Qdrant + Neo4j) |
| High-Stakes Actions | Full autonomy on writes/payments | Confidence thresholds + async human-in-the-loop approval queues |
1. Stop Paying LLM Latency for Deterministic Decisions
Before spinning up an orchestrator, separate your workflow into deterministic steps and reasoning steps. If a routing decision can be solved with a regex, a SQL lookup, or a lightweight embedding similarity check, keep the LLM out of the critical path.
Here is a lightweight pattern using Python and Pydantic to enforce deterministic routing before invoking an agent:
from enum import Enum
from pydantic import BaseModel, Field
class RouteTarget(str, Enum):
BILLING_SQL = "billing_sql"
TECH_RAG = "tech_rag"
HUMAN_ESCALATION = "human_escalation"
class RoutingDecision(BaseModel):
target: RouteTarget
confidence: float = Field(..., ge=0.0, le=1.0)
requires_approval: bool
def fast_gatekeeper(query: str) -> RoutingDecision | None:
"""Handle obvious deterministic routes in <5ms before calling an LLM."""
q = query.lower().strip()
if any(k in q for k in ["invoice", "receipt", "gst", "tax id"]):
return RoutingDecision(
target=RouteTarget.BILLING_SQL,
confidence=0.98,
requires_approval=False,
)
if any(k in q for k in ["cancel contract", "legal", "breach"]):
return RoutingDecision(
target=RouteTarget.HUMAN_ESCALATION,
confidence=1.0,
requires_approval=True,
)
return None # Fall back to LLM router only for ambiguous queries
This single gatekeeper layer typically cuts token costs by 35%–50% and drops median response times below 200ms for routine operational queries.
2. Enforce State Machines Over Free-Form Agent Loops
Giving five agents access to a shared group chat and hoping they converge is a recipe for infinite token burn. Instead, model your workflow as a Directed Acyclic Graph (DAG) or a bounded state graph (using frameworks like LangGraph or Temporal):
- Explicit State Schema: Define every variable an agent can read or mutate in a typed state object.
-
Hard Iteration Caps: Set
max_steps=5on every subgraph. If an agent hasn't resolved the task within five hops, fail gracefully and route to a fallback handler. - Idempotent Tools: Every external API call (CRM updates, email triggers, ERP entries) must accept an idempotency key so retries never create duplicate records.
3. Ground Retrieval with Hybrid Vector + Graph Context
Pure vector search struggles when queries require multi-hop relationships—for example, "Which enterprise clients on Plan B had an unresolved support ticket in the last 14 days?"
In our agentic AI development builds, we pair:
- Vector Search (Qdrant / pgvector): For unstructured documentation, SOPs, and call transcripts.
- Graph / Relational Lookups (Neo4j / PostgreSQL): For exact entity relationships, permissions, and account hierarchies.
Always attach source chunk IDs to every retrieved fact and require the synthesis agent to cite them in its structured output. If no chunk supports a claim, the model must return "insufficient_context" rather than guessing.
4. Production Readiness Checklist Before Launch
Before shipping any AI agent to live users or internal operations teams, verify these five controls:
- [ ] Structured Outputs Only: Zero regex scraping on markdown responses.
- [ ] Scoped Permissions: Read-only database credentials for retrieval agents; separate write-scoped service accounts behind approval gates.
- [ ] Prompt Injection Filtering: Input sanitization before user-supplied text enters tool-calling contexts.
- [ ] Traceability: Full token, latency, and tool-call logging per trace ID.
- [ ] Graceful Degradation: Automatic fallback to deterministic workflows if upstream model APIs rate-limit or time out.
About WTFearB
WTFearB (What The Fear, Boss / Business) is a custom AI software development and business automation agency. Guided by our core philosophy—Make What If Work—we engineer production-grade multi-agent AI systems, custom LLM workflows, and full-stack enterprise SaaS platforms for ambitious businesses.
Have a complex operational workflow you want to automate without fragile prototypes? Explore how we build at wtfearb.tech.
來源:DEV Community · dev.to