AutoGPT 智能體循環耗用 47GB 記憶體：凌晨故障的排查與修復

Notes from a 2 AM On-Call: When AutoGPT's Agent Loop Ate 47GB RAM (And the Debug Path That Actually Worked)

The Failure Symptom

[2026-01-19 02:14:33] INFO: Agent cycle 847 starting
[2026-01-19 02:14:51] WARNING: Memory usage: 18.2 GB
[2026-01-19 02:15:12] WARNING: Memory usage: 31.4 GB
[2026-01-19 02:15:19] CRITICAL: Memory usage: 47.1 GB
[2026-01-19 02:15:21] KILLED: OOM (signal 9, exit code 137)

Our AutoGPT instance was running a multi-day code analysis task. Around cycle 800, memory consumption spiked exponentially. Within 90 seconds, the container hit its 48GB limit and the kernel OOM killer intervened.

The AutoGPT agent wasn't crashing cleanly — no Python traceback, no graceful shutdown. Just a hard kill and 847 cycles of work lost.

The Red Herring

Standard AutoGPT documentation suggests monitoring with psutil and setting --max-iterations. We had both configured:

# autogpt_config.py
MAX_ITERATIONS = 1000
MEMORY_BACKEND = "json_file"  # not Redis
CONTINUOUS_MODE = False

Running py-spy during a reproduction attempt showed the expected profile: most time in OpenAI API calls, JSON parsing, and filesystem I/O. Nothing suspicious in the flame graph.

The misleading part: tracemalloc snapshots showed memory allocated per cycle was reasonable (~200MB peak). But total RSS kept climbing. The profiler was measuring allocations, not retention.

The Root Cause

AutoGPT's agent loop stores execution history in agent.history — a Python list that accumulates every tool call, LLM response, and intermediate reasoning step. In our case:

# Inside forge/agent.py (simplified)
class Agent:
    def __init__(self):
        self.history = []  # unbounded growth

    def execute_cycle(self):
        result = self.llm.complete(prompt)
        self.history.append({  # never pruned
            'cycle': self.cycle_count,
            'prompt': prompt,
            'response': result,
            'tools_used': self.tools_log,
            'timestamp': time.time()
        })

With MEMORY_BACKEND = "json_file", AutoGPT writes snapshots to disk but keeps the full in-memory history for context window reconstruction. After 847 cycles of multi-turn LLM conversations (each storing 4K-8K token prompts/responses), we had 6.2GB of JSON strings in RAM.

The actual killer: nested references. Each tools_log entry contained full file contents from code analysis tasks. One file read stored a 12MB Python repository snapshot. Multiply by 847 cycles.

The Runnable Fix

We implemented a sliding window with aggressive pruning:

#forge/agent.py patch
class Agent:
    def __init__(self, history_window=50):
        self.history = []
        self.history_window = history_window
        self.archive_path = Path("agent_archive")

    def execute_cycle(self):
        result = self.llm.complete(prompt)

        # Compress tool outputs before storing
        tools_summary = [
            {k: v if k != 'content' else f"<{len(v)} bytes>" 
             for k, v in tool.items()}
            for tool in self.tools_log
        ]

        self.history.append({
            'cycle': self.cycle_count,
            'prompt': prompt[:500],  # truncate prompts
            'response': result,
            'tools_used': tools_summary,
            'timestamp': time.time()
        })

        # Sliding window eviction
        if len(self.history) > self.history_window:
            archived = self.history.pop(0)
            self._archive_to_disk(archived)

Key changes:

1. Truncate prompts to 500 chars (full context reconstructed from disk archive if needed)
2. Replace file contents in tool logs with byte-length markers
3. Sliding window keeps only last 50 cycles in RAM (configurable via --history-window)
4. Archive evicted cycles to compressed JSONL for post-mortem analysis

After deploying this patch, the same 1000-cycle workload peaked at 4.1GB RSS and ran to completion.

Production Benchmark Against Multi-Provider Failover

When testing long-running agent tasks against different LLM routing strategies, we benchmarked AutoGPT with primary/fallback configuration through B-Lost's relay gateway:

# .env configuration
OPENAI_API_BASE=https://b-lost.com/v1
OPENAI_API_KEY=your_key

# Routing config enables automatic failover
# Primary: GPT-4, Fallback: Claude Sonnet

With this topology, when the primary model hit rate limits during cycle 340, the failover added 180ms latency but prevented cascade failures. The memory fix remained stable across both providers — the leak was architectural, not model-specific.

The Invariant Rule

Never trust in-memory collections in agent loops without explicit eviction policies.

Autonomous agents are designed to run indefinitely. Any unbounded data structure — history logs, cache dictionaries, event queues — will eventually OOM in production. If your agent framework doesn't expose --max-history-size or equivalent, patch it before the first long-running deployment.

The standard --max-iterations flag prevents infinite loops but does nothing for memory growth within bounded iteration counts. Our 847-cycle failure happened well under the 1000-iteration limit.

What We Actually Validated

# Reproduction test with patched agent
$ docker run --memory=8g autogpt:patched \
    --continuous \
    --history-window 30 \
    --task "analyze-codebase" \
    --max-iterations 2000

# Memory profile after 2000 cycles
RSS: 3.8 GB (stable)
Archive size: 1.2 GB compressed JSONL
Zero OOM kills over 72-hour stress test

The fix is now running in our staging cluster handling 12 concurrent AutoGPT instances with 16GB container limits. No OOM incidents in 9 days of continuous operation.

If you're running AutoGPT in production and seeing gradual memory growth, check len(agent.history) before profiling the usual suspects. The real leak is often in domain logic, not framework internals.