Agent 提示詞的上下文分流：術語、五條分支與無依賴預算檢查器

Consider a support-triage agent that reads a ticket, pulls four tool results, and asks a model to classify and draft a reply. It works locally. Then someone adds a fifth tool, the history buffer grows by one turn, and every call starts failing on length before the model ever sees the question. This is a composite scenario, not a logged incident — but it is the most common shape of the failure.

The tempting fix is "switch to a bigger window." That is also the fix that keeps you on a treadmill, because the window you get from a free model endpoint is the one constraint you do not control. The fix that survives is triage: decide, before the call, which spans of your prompt are load-bearing and which are recoverable.

This post is that decision procedure. Terms first, then a five-branch tree with a worked example at each leaf, then the code I use to make the decision mechanical instead of vibes-based.

1. Glossary: the six terms that do the work

Span. One named, independently removable block of text in the prompt: the system instruction, the ticket body, each tool result, each prior turn. Triage is impossible if your prompt is one concatenated string; it is trivial if it is a list of spans.

Hard window. The context limit the endpoint enforces. If you exceed it, the request fails or gets silently cut. Treat it as unknown until you read the documentation — do not infer it from a successful call.

Soft budget. The number you actually target, deliberately below the hard window, leaving room for the model's output and for tokenizer drift between your counter and the provider's.

Eviction class. The label that decides whether a span may be dropped. Three classes are enough: pinned (the task is meaningless without it), reconstructible (can be replaced by a stub or summary), recoverable (can be re-fetched on demand if the model asks).

Truncation debt. The information you removed that the model will later need. It is not free. A prompt that fits by dropping a tool result often produces a confident, wrong answer twenty seconds later.

Leaf. A terminal branch of the decision tree: one specific trimming action, plus the check that tells you the trim was safe.

2. The tree: five branches, five leaves

Ask the branches in order. The first one that matches is your leaf; stop there.

Branch 1 — Does one pinned span already exceed the soft budget?

If a single pinned span is too large, no amount of dropping history will save you. Chunk it.

Leaf 1A — Chunk and label. Split the span into numbered chunks that each fit alone, process chunk 1 first, and carry forward only the extracted fields, not the raw text.

Worked example: a 900-line log file attached to a bug ticket. Chunk by time window, ask for {error_signature, first_occurrence, affected_endpoint} from each chunk, then classify on the merged fields. The merge output is a few hundred tokens; the log never needs to re-enter the window.

Branch 2 — Is the overflow from accumulated conversation turns?

Leaf 2A — Windowed eviction with a carry-forward summary. Keep the last n turns verbatim, replace everything older with one summary span marked reconstructible, and cap the summary at a fixed token count so it cannot itself become the problem.

Worked example: twelve turns of a refactoring session. Keep turns 9–12 verbatim; compress 1–8 into a bullet list of decisions ("chose strategy B; rejected caching layer; rename pending"). The decisions are what future turns reference. The transcript is not.

Branch 3 — Is the overflow from tool output?

This is the branch people skip, and it is usually the right one.

Leaf 3A — Lossy projection at the tool boundary. Do not store what the tool returned; store what the next step needs. A grep result becomes a list of {path, line, match}, not the file. A JSON API response becomes the two fields the prompt actually names.

Worked example: a dependency-audit tool returns 4 KB of advisory JSON; the agent only needs (package, severity, patched_version). Project to that triple at the tool boundary and the span drops by an order of magnitude — with no loss, because the model was never going to read the rest.

Branch 4 — Is the overflow from the instruction block itself?

Leaf 4A — Deduplicate and demote. Long system prompts accumulate contradictory restatements. Split the instruction span into numbered rules, drop exact duplicates, and demote examples into a separate reconstructible span that is only included when the task is in the "hard case" set.

Worked example: an instruction block with three separate paragraphs explaining output JSON format. Collapse to one rule plus one schema block. If the schema is large, keep the schema and move the worked examples out.

Branch 5 — Nothing trims enough without cutting pinned content.

Leaf 5A — Reduce the unit of work. Do not shrink the prompt further; shrink the job. One call per file instead of one call per repository. This is where triage becomes planning, and it is a legitimate outcome, not a failure.

Worked example: "review this PR" against a 40-file diff. Convert to per-file review calls plus one aggregator call that receives only the per-file verdicts.

3. Make it mechanical: a budget checker with no dependencies

The tree is only useful if a machine applies it. This checker takes spans, applies eviction classes, and returns what survived and what was removed. The token counter is pluggable — the default character heuristic is a stand-in so the example runs without a tokenizer.

from dataclasses import dataclass

@dataclass
class Span:
    name: str
    text: str
    evict: str   # "pinned" | "reconstructible" | "recoverable"
    priority: int  # lower number = dropped earlier

def char_counter(text: str) -> int:
    return max(1, len(text) // 4)

def triage(spans, soft_budget, counter=char_counter):
    total = sum(counter(s.text) for s in spans)
    if total <= soft_budget:
        return list(spans), [], total

    droppable = sorted(
        (s for s in spans if s.evict != "pinned"),
        key=lambda s: (s.priority, -counter(s.text)),
    )
    kept, dropped = list(spans), []
    for span in droppable:
        if total <= soft_budget:
            break
        kept.remove(span)
        dropped.append(span)
        total -= counter(span.text)
    return kept, dropped, total

def render(kept, dropped):
    lines = [s.text for s in kept]
    for s in dropped:
        lines.append(f"[{s.name} omitted: {s.evict}]")
    return "\n\n".join(lines)

Run it on a synthetic fixture — counts below are fixture output from char_counter, not provider tokenizer numbers and not a benchmark:

spans = [
    Span("instructions", "..." * 200, "pinned", 0),
    Span("ticket",       "..." * 300, "pinned", 0),
    Span("tool_audit",   "..." * 900, "recoverable", 1),
    Span("history_1_8",  "..." * 700, "reconstructible", 2),
    Span("schema",       "..." * 250, "pinned", 0),
]
kept, dropped, total = triage(spans, soft_budget=1200)
print(total, [s.name for s in dropped])

The point of the render stub line is honesty: the model is told a span exists and was omitted, so it can ask for it back instead of assuming the absence means "nothing there." Silent removal is where truncation debt gets expensive.

4. Which leaf for which symptom

The table is the practical artifact. In practice, most teams are in branch 3 and believe they are in branch 2.

5. Limitations, and who should skip this

• Counters disagree. Your character heuristic and the provider's tokenizer will drift. That is why the soft budget exists; set it with margin, not to the last token.
• Summaries are lossy in ways tests miss. If your task depends on exact strings (IDs, hashes, error codes), chunk summarization is the wrong leaf — keep those spans pinned and reduce the job instead.
• This is not a cost optimization. Triage reduces failures; it does not necessarily reduce call count. Branch 5 in particular increases it.
• Skip this if your prompts fit comfortably with headroom, if you have a single fixed prompt with no history or tools, or if silent truncation by the endpoint is already documented and you accept it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

6. Where the free surfaces fit

The workflow above was written around two surfaces: a free server running the agent runtime (where span assembly and the budget check happen) and free model access at the endpoint (where the window is enforced). Keeping the triage step on the server side means the model endpoint only ever sees a prompt that already fits — the failure moves from "request rejected" to "span listed as omitted," which is a much cheaper failure to debug.

Two cautions. Free availability is not the same as unlimited capacity: read the current documentation for quotas, window sizes, and server limits rather than assuming last month's numbers still hold, and back off immediately on a 429 instead of retrying around it. And measure your own prompts — the fixture above tells you nothing about your traffic.

If you want to try the leaves against a live endpoint, the budget checker is the only thing you need to bring; MonkeyCode's free model access and free server option are the two surfaces this was written around, and the tree works the same without them.