LLM 帳單為何暴增？先檢查四個成本來源

The invoice arrives and it is bigger than last month. Traffic grew a little. The bill grew a lot.

The usual reaction is "we need a cheaper model." Sometimes that is right. More often, the money is leaking somewhere nobody is looking, and swapping models just moves the leak.

So here is this Friday's question: when your LLM bill jumps, which of these is the real culprit on your team?

A) Long prompts — context stuffing, giant system prompts, whole documents pasted in
B) Verbose outputs — answers far longer than anyone reads
C) Retries and agent loops — the same work paid for two, five, ten times
D) The wrong model tier — a frontier model doing classification a small model could handle

My lean: C and D are the silent ones. A and B show up the moment you look at a single request. C and D only show up when you look at the whole workflow.

A) Long prompts: you pay for every token, every call

Input tokens are cheap per token, but you pay for them on every single request. A 6,000-token system prompt that never changes is 6,000 tokens billed on call one and on call one million.

What to check:

Is the stable part of the prompt at the top so provider prompt caching can kick in? Both OpenAI and Anthropic document discounted cached input reads when the prefix is identical — see OpenAI prompt caching and Anthropic pricing.
Are you retrieving 20 chunks when the answer needs 3?
Is conversation history trimmed or summarized, or does it grow forever?

B) Verbose outputs: the expensive side of the price sheet

On most provider price sheets, output tokens cost several times more than input tokens. A model that "helpfully" writes 600 words when the UI shows 2 sentences is burning the most expensive tokens you buy.

What to check:

Do you set a sensible max_tokens per use case?
Does the prompt say what length and format you actually want?
For machine-consumed outputs, are you asking for structured JSON instead of prose that you then parse?

C) Retries and agent loops: paying for the same work again

This is the one that hides. A single user action can fan out into:

a validation failure that triggers a re-ask,
a provider timeout that triggers a retry,
an agent that calls a tool, gets a confusing result, and tries again — eight times.

Each step looks reasonable in isolation. The total is a multiple of what you budgeted.

What to check:

Cost per completed task, not per request. If a "summarize ticket" task averages four LLM calls, that is your real unit price.
A hard cap on agent iterations and on retries per request.
Whether retries are idempotent and logged, so you can see how often they fire.

Tracing tools make this visible. Langfuse token and cost tracking attributes cost per trace and per span, and LiteLLM cost tracking gives spend per key, team, or model if you route through its proxy.

D) The wrong model tier: a sledgehammer for every nail

Many production calls are boring: classify this, extract these fields, route this ticket. Those rarely need the biggest model you have access to.

What to check:

List your LLM call sites and tag each one: reasoning-heavy vs routine.
Run your eval set on a smaller model for the routine ones. If quality holds, switch.
For work that is not latency sensitive, look at batch endpoints — both OpenAI Batch and Anthropic Message Batches are priced below their synchronous APIs.

OpenAI's latency optimization guide makes a related point: fewer tokens, fewer requests, and smaller models where possible help both speed and spend.

A 30-minute audit you can run this week

Pull one day of traces or logs.
Group cost by task, not by request.
For the top three tasks, write down: input tokens, output tokens, calls per task, model used.
The biggest number on that sheet is your answer to A, B, C, or D.

Nine times out of ten, the fix is not a new vendor. It is a cap, a cache-friendly prompt order, or a smaller model on the routine path.

Your turn

When your LLM bill jumped, what was it really?

A) Long prompts, B) Verbose outputs, C) Retries and agent loops, or D) The wrong model tier?

Drop your letter in the comments — and if it was something else entirely, I would love to hear it.