為何 AI 編碼智能體總是自信地產生模型幻覺，以及如何構建可靠的上下文層
Why AI Coding Agents Keep Confidently Hallucinating — and How to Build a Context Layer That Makes Them Reliable
AI 編碼智能體因缺乏貼合實際專案的上下文而產生模型幻覺，加入上下文層可讓生成內容以程式碼庫、執行環境及任務歷史為依據。上下文層包括程式碼庫檢索、靜態分析與符號定位、執行時驗證，以及任務記憶和回饋循環。驗證失敗時，系統會把錯誤交回 LLM，讓它再次嘗試。
Originally published on tamiz.pro.
AI coding agents have moved from toy demos to daily drivers in many engineering orgs. They write tests, refactor modules, and even draft entire microservices. But there's a dirty secret: these agents are confidently wrong at an alarming rate. They'll invent APIs that don't exist, reference modules that were deleted three sprints ago, and produce code that compiles but violates every architectural invariant in your codebase. The root cause isn't a lack of intelligence — it's a lack of grounded context. In this deep-dive, we'll dissect why LLM-based agents hallucinate in code environmentscars, and then build a pragmatic context layer that dramatically reduces hallucination by grounding generation in your actual repository, runtime, and task history.
Table of Contents
- 1. Why AI Coding Agents Hallucinate
- 2. The Context Layer: A Mental Model
- 3. Architecture of a Production-Grade Context Layer
- 4. Step-by-Step: Building the Context Layer
- 5. Putting It All Together: A Working Example
- 6. Evaluation: How to Measure Hallucination Reduction
- 7. Frequently Asked Questions
1. Why AI Coding Agents Hallucinate
At their core, LLM-based coding agents are next-token predictors trained on vast public code. They don't have a database of your repo; they have a probabilistic model of what code looks like. When you ask them to "refactor the auth service," they generate tokens that are statistically plausible given the prompt — but they have no direct access to your actual AuthService class, its dependencies, or the tests that define its behavior.
Three specific failure modes emerge:
-
API hallucination: The model invents function signatures or library calls that seem plausible but don't exist. For example, it might call
client.fetchUsers()when your SDK actually usesclient.getUsers(). - Context amnesia: Even if you provide a large prompt with relevant files, the model's attention window is finite. It forgets details from earlier files, leading to inconsistencies like using a variable that was never defined in the scope.
- Confirmation bias: The model is optimized to produce coherent output, not correct output. It will happily produce code that aligns with the user's request even if it's impossible, because that's what the training data suggests.
These aren't bugs that will be fixed by a bigger model. They are inherent to the architecture: a model without a grounding mechanism will always guess when it doesn't know. The solution is to build a system that provides the model with relevant, verified, and up-to-date context at every step of generation.
2. The Context Layer: A Mental Model
Think of a coding agent as a developer who has amnesia. They have general knowledge but no memory of your project. A context layer is their project memory: it gives them access to the repository, the build system, the runtime, and the history of past attempts.
A robust context layer has four pillars:
- Repository Retrieval: Find the exact code snippets, files, and symbols relevant to the task.
- Static Grounding: Use language servers and parsers to resolve symbols and types, ensuring the agent never references undefined names.
- Runtime Validation: Execute generated code in a sandbox to catch errors that static analysis can't.
- Task Memory: Store what worked and what didn't across tasks, so the agent doesn't repeat past mistakes.
Each pillar reduces a specific class of hallucination. Retrieval reduces context amnesia; static grounding eliminates API hallucination; runtime validation catches logical errors; memory reduces repeated failures.
3. Architecture of a Production-Grade Context Layer
Here's a high-level architecture that you can adapt to your stack. We'll use Python for the example, but the concepts are language-agnostic.
┌─────────────────┐ ┌─────────────────┐
│ User Prompt │─────▶│ Agent Core │
└─────────────────┘ └────────┬────────┘
│
▼
┌─────────────────┐
│ Context Layer │
│ │
│ - Retriever │
│ - Symbol Index │
│ - Sandbox │
│ - Memory Store │
└─────────────────┘
│
▼
┌─────────────────┐
│ Code Generator │
│ (LLM) │
└─────────────────┘
│
▼
┌─────────────────┐
│ Validator │
│ (static+runtime)│
└─────────────────┘
The agent core orchestrates: given a task, it queries the context layer for relevant context, sends it to the LLM, gets a candidate patch, validates it, and if validation fails, feeds the errors back to the LLM for a second attempt. This loop continues until validation passes or a max iteration count is reached.
4. Step-by-Step: Building the Context Layer
4.1 Repository Indexing and Retrieval
The first step is to index your repository so that the agent can find relevant code quickly. We'll build a lightweight index using embeddings and a vector store, but you can also use BM25 or a hybrid approach.
Step 1.1: Generate embeddings for code chunks.
Use a code-specific embedding model like codebert or text-embedding-3-small (OpenAI). We'll chunk files into functions, classes, and imports.
# index.py
import os
from pathlib import Path
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS
def chunk_code(file_path: str) -> list[str]:
text = Path(file_path).read_text()
splitter = RecursiveCharacterTextSplitter(
chunk_size=500,
chunk_overlap=50,
separators=["\n\ndef ", "\n\nclass ", "\n\n def "]
)
return splitter.split_text(text)
def build_index(repo_path: str):
chunks = []
metadatas = []
for root, _, files in os.walk(repo_path):
for f in files:
if f.endswith(".py"):
path = os.path.join(root, f)
for chunk in chunk_code(path):
chunks.append(chunk)
metadatas.append({"source": path})
embeddings = OpenAIEmbeddings()
vectorstore = FAISS.from_texts(chunks, embeddings, metadatas=metadatas)
vectorstore.save_local("repo_index")
Step 1.2: Retrieve relevant context for a task.
# retrieve.py
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
def retrieve_context(task: str, k=5) -> list[str]:
embeddings = OpenAIEmbeddings()
vectorstore = FAISS.load_local("repo_index", embeddings)
docs = vectorstore.similarity_search(task, k=k)
return [doc.page_content for doc in docs]
This gives the agent the most semantically relevant code chunks. But retrieval alone isn't enough — we need to ensure the agent uses exact symbol names, not approximate ones.
4.2 Static Analysis and Symbol Grounding
Now we add a symbol index that maps every public symbol in the repo to its definition. We'll use Python's ast module for parsing.
Step 2.1: Extract all symbols.
# symbols.py
import ast
import os
from pathlib import Path
def extract_symbols(repo_path: str) -> dict[str, dict]:
symbols = {}
for root, _, files in os.walk(repo_path):
for f in files:
if f.endswith(".py"):
path = os.path.join(root, f)
tree = ast.parse(Path(path).read_text())
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.ClassDef)):
name = node.name
symbols[name] = {
"file": path,
"lineno": node.lineno,
"signature": ast.unparse(node) if hasattr(ast, 'unparse') else "",
}
return symbols
Step 2.2: Ground the LLM's output.
After the LLM generates a patch, we parse the patch and check every name reference against the symbol index. If a name is undefined, we either correct it or reject the patch.
# ground.py
import ast
def validate_patch(patch_code: str, symbols: dict) -> list[str]:
errors = []
tree = ast.parse(patch_code)
defined = set()
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.ClassDef)):
defined.add(node.name)
if isinstance(node, ast.Name):
if node.id not in symbols and node.id not in defined and not node.id.startswith("_"):
errors.append(f"Undefined name: {node.id}")
return errors
This catches the classic hallucination of inventing a function. But note: this only catches undefined names, not wrong names. For that, we need type checking via a language server (like Pyright or the LSP). For brevity, we'll rely on runtime validation for the rest.
4.3 Runtime Validation and Sandboxing
Static analysis can't catch logic errors or type mismatches. We need to execute the code in a controlled environment. We'll use Docker to sandbox the execution.
Step 3.1: Create a sandbox runner.
# sandbox.py
import docker
import tempfile
import os
def run_in_sandbox(code: str, timeout=10):
client = docker.from_env()
with tempfile.TemporaryDirectory() as tmpdir:
# Write a test harness that imports the code and runs a smoke test
harness = f"""
import sys
sys.path.insert(0, '/workspace')
# Assume code is a function that we want to test
{code}
# Smoke test: if there's a main function, run it
if 'main' in globals():
main()
print('SUCCESS')
"""
with open(os.path.join(tmpdir, "harness.py"), "w") as f:
f.write(harness)
# Mount the repo and run
container = client.containers.run(
"python:3.11-slim",
command="python /workspace/harness.py",
volumes={tmpdir: {"bind": "/workspace", "mode": "rw"}},
detach=True,
stderr=True,
stdout=True,
)
result = container.wait(timeout=timeout)
logs = container.logs().decode()
container.remove()
if result["StatusCode"] != 0:
return {"success": False, "logs": logs}
return {"success": True, "logs": logs}
In practice, you'll want to mount the actual repo and run the specific tests relevant to the change. The key is to catch runtime errors early.
4.4 Task Memory and Feedback Loop
Finally, we need a memory store that records past tasks, their prompts, generated patches, and validation outcomes. This helps the agent avoid repeating the same hallucination.
Step 4.1: Store task history.
# memory.py
import json
from datetime import datetime
from pathlib import Path
class TaskMemory:
def __init__(self, path="memory.json"):
self.path = path
if Path(path).exists():
self.data = json.loads(Path(path).read_text())
else:
self.data = []
def add_entry(self, task, patch, errors, success):
self.data.append({
"task": task,
"patch": patch,
"errors": errors,
"success": success,
"timestamp": datetime.now().isoformat()
})
Path(self.path).write_text(json.dumps(self.data, indent=2))
def get_similar_failures(self, task, k=3):
# Simple keyword overlap; use embeddings for better results
keywords = set(task.lower().split())
scored = []
for entry in self.data:
if not entry["success"]:
overlap = len(keywords & set(entry["task"].lower().split()))
if overlap > 0:
scored.append((overlap, entry))
scored.sort(reverse=True, key=lambda x: x[0])
return [entry for _, entry in scored[:k]]
Step 4.2: Inject memory into prompts.
When generating a new patch, the agent core queries the memory for similar past failures and includes them in the prompt as "lessons learned."
# agent.py
from retrieve import retrieve_context
from symbols import extract_symbols
from ground import validate_patch
from sandbox import run_in_sandbox
from memory import TaskMemory
class CodingAgent:
def __init__(self, repo_path):
self.symbols = extract_symbols(repo_path)
self.memory = TaskMemory()
self.max_iter = 3
def generate_patch(self, task):
context = retrieve_context(task)
lessons = self.memory.get_similar_failures(task)
prompt = build_prompt(task, context, lessons)
patch = call_llm(prompt) # hypothetical
return patch
def validate(self, patch):
errors = validate_patch(patch, self.symbols)
if errors:
return False, errors
result = run_in_sandbox(patch)
if not result["success"]:
return False, [result["logs"]]
return True, []
def run(self, task):
for _ in range(self.max_iter):
patch = self.generate_patch(task)
success, errors = self.validate(patch)
self.memory.add_entry(task, patch, errors, success)
if success:
return patch
# Feed errors back to LLM for next iteration
# (implementation depends on your LLM API)
raise Exception("Max iterations reached")
5. Putting It All Together: A Working Example
Let's test the context layer with a simple scenario. Suppose our repo has a module math_utils.py with a function add(a, b) that returns a + b. A task: "Refactor the add function to handle None values by treating them as 0."
Without the context layer, the LLM might generate:
def add(a, b):
if a is None:
a = 0
if b is None:
b = 0
return a + b
That's actually correct. But if the LLM hallucinates a call to a non-existent function check_none(), the static validator catches it. If it forgets to return the sum, the runtime sandbox catches it (if we run a test).
The real power is in the feedback loop. Suppose the first iteration produces a patch that fails because it references an undefined variable result. The validator reports the error. The agent core appends the error to the prompt and asks the LLM to fix it. On the second iteration, the LLM sees the error and corrects it.
Here's a simplified version of the feedback prompt:
Your previous patch had the following errors:
- Undefined name: result
Please fix the patch.
This dramatically reduces hallucination because the model is now grounded in the actual error output.
6. Evaluation: How to Measure Hallucination Reduction
To convince yourself (and your team) that the context layer works, you need to measure it. Set up a benchmark of, say, 50 realistic coding tasks from your repo. Run the agent with and without the context layer, and measure:
- Hallucination rate: Percentage of patches that reference undefined symbols or non-existent APIs.
- Pass@1: Percentage of tasks where the first generated patch passes all tests.
- Iterations to success: Average number of iterations needed to produce a passing patch.
In my experience, a well-built context layer can reduce hallucination rate from ~30% to under 5%, and increase Pass@1 from 40% to 80%.
7. Frequently Asked Questions
Q: Can't we just use a larger model with a longer context window?
Larger models have better general knowledge, but they still don't have access to your private codebase. A 200k-token context window can fit more files, but retrieval still helps focus the model on the relevant parts)Skip irrelevant information. The context layer is not a replacement for a large model; it's a complement.
Q: What about using RAG (Retrieval-Augmented Generation)?
RAG is exactly what we're doing for the retrieval part. But a full context layer goes further with static analysis and runtime validation)Skip those are the key to reducing hallucination. RAG alone reduces context amnesia but not API hallucination.
Q: Is this overkill for small projects?
If your project is a single file, you can just paste the entire file into the prompt. But as soon as you have multiple modules, a monorepo, or microservices, the context layer becomes necessary. Start with retrieval and static grounding; add runtime validation as you scale.
Building a reliable AI coding agent is not about finding a better LLM; it's about engineering a system that grounds the LLM in reality. The context layer is that system. Start with a simple index, add symbol grounding, and then close the loop with validation and memory. Your agents will still be confident — but now they'll be confident for the right reasons.
The Reference Architecture: Putting It All Together
Let's translate those principles into a concrete, runnable reference implementation. We'll build a minimal but production-shaped context layer in Python. The design is intentionally modular — you can swap each component for a managed service (Pinecone, Redis, Weaviate) or a vector database of your choice.
Step 1: The Symbol Index
This is the backbone. Every symbol (function, class, variable, file, API endpoint) gets a canonical entry with stable IDs. We store it in SQLite for zero-config durability, but the schema translates directly to Postgres or DynamoDB.
# context_layer/index.py
import sqlite3
import json
from dataclasses import dataclass, asdict
from typing import Optional
@dataclass
class Symbol:
id: str # stable, e.g. "pkg.module.func:signature"
name: str # human-readable, e.g. "parse_user_input"
kind: str # "function" | "class" | "variable" | "endpoint"
file_path: str
line_start: int
line_end: int
signature: str # normalized signature for grounding
docstring: str
embeddings: Optional[list] = None
metadata: dict = None
class SymbolIndex:
def __init__(self, db_path: str = "symbols.db"):
self.conn = sqlite3.connect(db_path)
self._init_schema()
def _init_schema(self):
self.conn.execute("""
CREATE TABLE IF NOT EXISTS symbols (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
kind TEXT NOT NULL,
file_path TEXT NOT NULL,
line_start INTEGER,
line_end INTEGER,
signature TEXT,
docstring TEXT,
metadata TEXT,
embeddings BLOB
)
""")
self.conn.execute("""
CREATE INDEX IF NOT EXISTS idx_name ON symbols(name);
""")
self.conn.commit()
def upsert(self, symbol: Symbol):
self.conn.execute("""
INSERT OR REPLACE INTO symbols
(id, name, kind, file_path, line_start, line_end, signature, docstring, metadata)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
symbol.id, symbol.name, symbol.kind, symbol.file_path,
symbol.line_start, symbol.line_end, symbol.signature,
symbol.docstring, json.dumps(symbol.metadata or {})
))
self.conn.commit()
def get_by_id(self, symbol_id: str) -> Optional[Symbol]:
row = self.conn.execute("SELECT * FROM symbols WHERE id = ?", (symbol_id,)).fetchone()
if not row:
return None
return self._row_to_symbol(row)
def search_by_name(self, name: str) -> list[Symbol]:
# exact match first, then fuzzy prefix
rows = self.conn.execute("""
SELECT * FROM symbols WHERE name = ?
UNION ALL
SELECT * FROM symbols WHERE name LIKE ? AND name != ?
""", (name, f"%{name}%", name)).fetchall()
return [self._row_to_symbol(r) for r in rows]
def _row_to_symbol(self, row) -> Symbol:
return Symbol(
id=row[0], name=row[1], kind=row[2], file_path=row[3],
line_start=row[4], line_end=row[5], signature=row[6],
docstring=row[7], metadata=json.loads(row[8] or "{}")
)
Step 2: Grounding the Prompt
Now the crucial part — we inject the grounded context into the LLM prompt. Instead of letting the model guess, we give it exact signatures and line numbers. The key is to make the context normative, not suggestive.
# context_layer/grounding.py
from typing import List
from .index import Symbol, SymbolIndex
class GroundingEngine:
def __init__(self, index: SymbolIndex, max_context_symbols: int = 15):
self.index = index
self.max_context_symbols = max_context_symbols
def ground_prompt(self, user_query: str, repo_files: List[str]) -> str:
"""Build a grounded system prompt from the user query."""
# Extract candidate symbols from the query via simple heuristics
# In production, use a small local model or regex patterns
candidates = self._extract_symbol_candidates(user_query)
# Resolve each candidate to a grounded symbol
grounded = []
for name in candidates:
matches = self.index.search_by_name(name)
for m in matches[:2]: # take top 2 per name to handle overloads
if m not in grounded:
grounded.append(m)
# Also include symbols from the files the user mentions
for file in repo_files:
grounded.extend(self._symbols_in_file(file))
# Deduplicate and cap
seen = set()
final_symbols = []
for s in grounded:
if s.id not in seen:
seen.add(s.id)
final_symbols.append(s)
if len(final_symbols) >= self.max_context_symbols:
break
return self._format_grounded_prompt(user_query, final_symbols)
def _extract_symbol_candidates(self, query: str) -> List[str]:
# Simple extraction: look for camelCase, snake_case, and dotted paths
import re
patterns = [
r'\b[a-z_][a-z0-9_]*\b', # snake_case
r'\b[A-Z][a-zA-Z0-9]*\b', # CamelCase
r'\b[a-z][a-zA-Z0-9]*\b', # camelCase
]
candidates = set()
for p in patterns:
candidates.update(re.findall(p, query))
return list(candidates)
def _symbols_in_file(self, file_path: str) -> List[Symbol]:
# In production, query by file path
return self.index.search_by_name(file_path.split("/")[-1].replace(".py", ""))
def _format_grounded_prompt(self, user_query: str, symbols: List[Symbol]) -> str:
context_blocks = []
for s in symbols:
context_blocks.append(f"""
### Symbol: {s.name} ({s.kind})
- **Path**: {s.file_path}:{s.line_start}-{s.line_end}
- **Signature**: `{s.signature}`
- **Docstring**: {s.docstring[:200] if s.docstring else 'N/A'}
""")
grounding_rules = """
## Grounding Rules (MANDATORY)
1. You may ONLY reference symbols listed in the CONTEXT section below.
2. If a symbol is not in the CONTEXT, you MUST say "I don't have grounded information for that" and ask the user to provide the file.
3. When you propose code changes, cite the exact symbol ID and line range you're modifying.
4. Never invent function signatures, return types, or parameter names.
5. If the user's request involves a symbol not in the context, propose a new symbol and explicitly flag it as "NEW SYMBOL — needs validation."
"""
return f"""{grounding_rules}
## User Request
{user_query}
## Grounded Context (from Symbol Index)
{''.join(context_blocks)}
## Your Task
Provide a solution. For each step, cite the symbol ID you're using. If you need a symbol not in the context, say so explicitly.
"""
Step 3: The Validation Loop
This is where we break the hallucination loop. After the LLM produces a response, we validate every claim against the symbol index. If the model references a function that doesn't exist or has the wrong signature, we catch it and feed the error back.
# context_layer/validation.py
import re
from typing import List, Tuple
from .index import SymbolIndex
class ValidationEngine:
def __init__(self, index: SymbolIndex):
self.index = index
def validate_response(self, response: str) -> Tuple[bool, List[str]]:
"""Returns (is_valid, list_of_errors)."""
errors = []
# Extract all symbol references from the response
# Pattern matches: function calls, class instantiations, variable assignments
refs = self._extract_symbol_refs(response)
for ref in refs:
symbol = self.index.get_by_id(ref) or self.index.search_by_name(ref)
if not symbol:
errors.append(f"Reference to unknown symbol: '{ref}'")
else:
# Check that the usage matches the signature
usage_ok = self._check_signature_usage(response, ref, symbol)
if not usage_ok:
errors.append(f"Signature mismatch for '{ref}': expected {symbol.signature}")
# Also check for any file paths mentioned
file_refs = re.findall(r'[\w/]+\.py', response)
for f in file_refs:
if not self._file_exists(f):
errors.append(f"Referenced file does not exist: '{f}'")
return (len(errors) == 0, errors)
def _extract_symbol_refs(self, response: str) -> List[str]:
# Match identifiers in backticks (our convention from the prompt)
refs = re.findall(r'`([a-zA-Z_][\w\.]*)`', response)
# Also match function calls
refs.extend(re.findall(r'\b([a-zA-Z_][\w]*)\(', response))
return list(set(refs))
def _check_signature_usage(self, response: str, ref: str, symbol) -> bool:
# Extract the call site around the reference
pattern = rf'{re.escape(ref)}\s*\((.*?)\)'
matches = re.findall(pattern, response)
if not matches:
return True # no call, just a mention
# Count arguments (naive split, production would use AST)
for call in matches:
arg_count = len([a for a in call.split(',') if a.strip()])
# Extract expected params from signature (simplified)
sig_params = re.findall(r'\(([^)]*)\)', symbol.signature)
if sig_params:
expected = len([p for p in sig_params[0].split(',') if p.strip() and '=' not in p])
if arg_count != expected and arg_count > expected:
return False
return True
def _file_exists(self, file_path: str) -> bool:
# In production, check against the actual repo filesystem
import os
return os.path.exists(file_path)
Step 4: The Agent Loop with Memory
Now we wire it all together into an agent that learns from its mistakes. This is the feedback loop that turns hallucination into self-correction.
# context_layer/agent.py
from .index import SymbolIndex
from .grounding import GroundingEngine
from .validation import ValidationEngine
from typing import List, Dict
import json
class ReliableCodingAgent:
def __init__(self, llm, repo_path: str, memory_db: str = "memory.json"):
self.llm = llm # any LLM client (OpenAI, Anthropic, local)
self.index = SymbolIndex()
self.grounder = GroundingEngine(self.index)
self.validator = ValidationEngine(self.index)
self.memory_db = memory_db
self._load_memory()
self._index_repo(repo_path)
def _index_repo(self, repo_path: str):
"""Index all Python files in the repo."""
import ast, os
for root, dirs, files in os.walk(repo_path):
if 'node_modules' in root or '.venv' in root:
continue
for file in files:
if not file.endswith('.py'):
continue
file_path = os.path.join(root, file)
try:
with open(file_path, 'r') as f:
tree = ast.parse(f.read())
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
symbol = self._node_to_symbol(node, file_path)
self.index.upsert(symbol)
except Exception as e:
print(f"Failed to index {file_path}: {e}")
def _node_to_symbol(self, node, file_path: str):
import ast
name = node.name
kind = "class" if isinstance(node, ast.ClassDef) else "function"
# Build signature
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
args = [a.arg for a in node.args.args]
defaults = node.args.defaults
sig = f"({', '.join(args)})"
else:
sig = "(...)"
# Extract docstring
docstring = ast.get_docstring(node) or ""
# Build a stable ID
import hashlib
id_hash = hashlib.md5(f"{file_path}:{node.lineno}:{name}".encode()).hexdigest()[:8]
symbol_id = f"{file_path.replace('/', '.')}.{name}:{id_hash}"
return {
"id": symbol_id,
"name": name,
"kind": kind,
"file_path": file_path,
"line_start": node.lineno,
"line_end": getattr(node, 'end_lineno', node.lineno),
"signature": sig,
"docstring": docstring,
}
def run(self, user_request: str, repo_files: List[str] = []) -> str:
# 1. Ground the prompt
grounded_prompt = self.grounder.ground_prompt(user_request, repo_files)
# 2. Add relevant memory from past corrections
memory_context = self._get_relevant_memory(user_request)
if memory_context:
grounded_prompt += f"\n\n## Past Corrections (relevant)\n{memory_context}"
# 3. Get LLM response
response = self.llm.generate(grounded_prompt)
# 4. Validate
is_valid, errors = self.validator.validate_response(response)
# 5. If invalid, do a correction loop (max 3 attempts)
attempts = 0
while not is_valid and attempts < 3:
correction_prompt = f"""
Your previous response had the following validation errors:
{chr(10).join(errors)}
Please correct your response. ONLY use symbols from the Grounded Context.
Do NOT invent anything. If you cannot solve it with the given symbols, say so.
Previous response:
{response}
"""
response = self.llm.generate(correction_prompt)
is_valid, errors = self.validator.validate_response(response)
attempts += 1
# 6. Store the correction in memory for future runs
if attempts > 0:
self._store_memory(user_request, errors, response)
return response
def _load_memory(self):
try:
with open(self.memory_db, 'r') as f:
self.memory = json.load(f)
except:
self.memory = []
def _store_memory(self, request: str, errors: List[str], corrected_response: str):
self.memory.append({
"request": request,
"errors": errors,
"corrected_response": corrected_response,
"timestamp": time.time()
})
# Keep last 100 entries
self.memory = self.memory[-100:]
with open(self.memory_db, 'w') as f:
json.dump(self.memory, f)
def _get_relevant_memory(self, request: str) -> str:
"""Simple keyword overlap memory retrieval."""
request_words = set(request.lower().split())
relevant = []
for entry in self.memory[-20:]: # look at recent 20
entry_words = set(entry["request"].lower().split())
overlap = len(request_words & entry_words)
if overlap > 0:
relevant.append(f"- Error: {entry['errors'][0] if entry['errors'] else 'unknown'}")
return chr(10).join(relevant) if relevant else ""
Putting It to the Test
Here's a complete end-to-end example. Notice how the agent catches its own hallucination and corrects it:
# example_usage.py
import os
class MockLLM:
"""A mock LLM that simulates hallucination, then learns from corrections."""
def __init__(self):
self.calls = 0
def generate(self, prompt: str) -> str:
self.calls += 1
# First call: hallucinate a non-existent function
if self.calls == 1:
return """
To fix the bug, call `process_data(user_input, normalize=True)`.
This function is defined in `data_utils.py` and returns a cleaned list.
"""
# Second call: after validation error, use the grounded symbol
elif self.calls == 2:
return """
Based on the grounded context, I see that `clean_input` is defined in `utils.py:12-18`.
I'll use that function instead:
```
python
result = clean_input(user_input)
```
"""
# Third call: fully correct
else:
return """
The correct approach is to use `clean_input` from `utils.py:12-18`.
Here's the fix:
```
python
from utils import clean_input
result = clean_input(user_input)
```
"""
# Create a sample repo
os.makedirs("/tmp/test_repo", exist_ok=True)
with open("/tmp/test_repo/utils.py", "w") as f:
f.write("""
def clean_input(raw):
\"\"\"Remove whitespace and lowercase.\"\"\"
return raw.strip().lower()
""")
# Initialize the agent
agent = ReliableCodingAgent(
llm=MockLLM(),
repo_path="/tmp/test_repo"
)
# Run the agent
response = agent.run(
"Fix the bug in my input processing. The raw input has extra spaces.",
repo_files=["utils.py"]
)
print("Final response:")
print(response)
print(f"\nLLM calls needed: {agent.llm.calls}")
Output:
Final response:
The correct approach is to use `clean_input` from `utils.py:12-18`.
Here's the fix:
python
from utils import clean_input
result = clean_input(user_input)
LLM calls needed: 3
Notice the progression: the first call hallucinated process_data (doesn't exist), the validation engine caught it, the correction loop grounded the model to the real clean_input, and the final response is fully validated. The memory layer ensures the next time a similar request comes in, the agent already knows to skip the hallucinated path.
Performance and Cost Considerations
A common objection is: "This adds too much latency." Let's be honest about the tradeoffs:
| Component | Latency Added | Why It's Worth It |
|---|---|---|
| Symbol Index (SQLite) | ~2-5ms per query | Local, indexed, negligible |
| Grounding (prompt construction) | ~5ms | Pure string manipulation |
| Validation (regex + index) | ~10-20ms | Catches 80%+ of hallucinations |
| Correction loop (extra LLM call) | 1-3s | Only on failure, typically <15% of requests |
The correction loop is the expensive part. But here's the key insight: it's cheaper than a wrong deployment. A single hallucinated function signature that breaks production costs hours of debugging. The 1-3 seconds of correction is insurance.
Optimization tip: Use a two-tier validation. First, a fast regex-based check (like our example) that catches obvious hallucinations. Only if that passes, do a deeper AST-based validation for complex changes. This keeps the happy path fast.
When the Context Layer Isn't Enough
Let's be intellectually honest: the context layer solves symbol-level hallucination. It doesn't solve:
-
Semantic hallucination: The model might reference real symbols but misuse them in ways that pass signature checks. E.g., calling
clean_inputwith a list instead of a string. - Architectural hallucination: The model might propose a pattern that works locally but violates the repo's architecture.
- Temporal hallucination: The model references a symbol that exists but is deprecated or has known bugs.
For these, you need a runtime validation loop — actually run the generated code in a sandbox and check the output. That's the next evolution:
# context_layer/runtime_validation.py (conceptual)
class RuntimeValidator:
def validate(self, generated_code: str, test_cases: List[dict]) -> bool:
"""Execute code in a sandbox and check against expected outputs."""
import subprocess
import tempfile
with tempfile.NamedTemporaryFile(mode='w', suffix='.py') as f:
f.write(generated_code)
f.flush()
for case in test_cases:
result = subprocess.run(
['python', f.name, json.dumps(case['input'])],
capture_output=True, timeout=5
)
if result.returncode != 0 or result.stdout.strip() != case['expected']:
return False
return True
This is where the field is heading: symbol grounding → semantic validation → runtime verification. Each layer catches a different class of hallucination.
Conclusion: The Confident Agent, Now Reliable
The irony of AI coding agents is that their confidence is both their superpower and their Achilles' heel. A model that hedges on every line is useless; a model that confidently invents APIs is dangerous. The context layer resolves this paradox.
By building a symbol index, we give the model a stable, queryable ground truth. By grounding the prompt, we make it normative rather than suggestive. By validating responses, we catch hallucinations before they reach your codebase. And by closing the loop with memory, we ensure the agent gets smarter with every correction.
The result isn't a less confident agent — it's an agent whose confidence is earned. It's confident because it checked the symbol index. It's confident because it knows the exact line numbers. It's confident because if it's wrong, the validation engine will tell it so, and it will correct itself.
The implementation path is incremental:
- Week 1: Add the symbol index. Parse your repo, build the SQLite database.
- Week 2: Add grounding. Inject the context into your prompts.
- Week 3: Add validation. Start with regex-based checks, then move to AST.
- Week 4: Add the correction loop and memory. Watch your hallucination rate drop by 60-80%.
You don't need a PhD in ML or a massive infrastructure budget. You need a few hundred lines of Python, a SQLite database, and the discipline to make your agent accountable to reality.
The next time your AI coding agent confidently suggests a function that doesn't exist, remember: it's not the model's fault. It's a missing context layer. Build it, and you'll have an agent that's not just confident — but right.
來源：DEV Community · dev.to