作者提出尚未有人聘用的「LLM 會計師」職務，專責追蹤模型成本

*Every month, a company that builds on language models receives an invoice from its model provider. It is usually one large number, sometimes with a split by model. Then someone in a meeting asks what the number bought, and nobody can answer.
*
That gap is the reason for a job that does not have a name yet. I am proposing one: the LLM accountant. The nearest established term is AI FinOps, but the work is closer to accounting than to infrastructure. It is about tracing every unit of spend to its cause, reconciling it against the invoice, explaining why it moved, and being able to prove it when someone asks.

Why this role should exist
Cost now comes from behavior, not capacity. Traditional cloud cost follows servers and traffic. LLM cost follows decisions made by software: how much context to send, which model to call, whether to retry, how many tool results to feed back. A prompt edit is a spending decision. A retry rule is a spending decision. Nobody approves these as spending decisions today.

One request can cost fifty times another. Two users can trigger the same feature, and one costs a few cents while the other costs tens of euros because a tool timed out and the agent looped. Averages hide this completely.

The invoice cannot answer the real questions. Which feature, which step, which customer, which prompt version? Provider dashboards give totals. The questions that save money need cost attached to context.

Some customers cost more than they pay. Without cost by tenant, nobody knows which ones, and pricing stays a guess.

Other people are starting to ask. Finance wants forecasts. Customers on usage-based plans want to know why their bill moved. Auditors and regulators ask how you know what an automated decision cost and why it happened. Someone must own those answers.

Nobody owns it today. Engineers see the symptoms, finance sees the total, and product sees neither. The work falls between them and gets done in a panic when the bill jumps.

That is the shape of every accounting role in history: money moving through a system nobody can fully see, and a person whose job is to make it legible.

What the LLM accountant owns
The chart of accounts: how spend is sliced into feature, agent step, model, tenant, and prompt version.
The ledger: a record of every model call with its token counts and context.
The rate card: dated prices per model, kept as history.
Reconciliation: monthly matching of the ledger to the provider invoice.
Variance analysis: explaining why any cost head moved.
Recommendations: a fix and an estimated saving for every finding.
Controls: per-session limits, spend alerts, and budgets.
Evidence: being able to show where any number came from.
What they should calculate, and why
A short list of numbers does most of the work. Each answers a question someone will eventually ask.

CalculationHowWhy it mattersCost per sessionTotal cost divided by sessionsNormalizes spend against trafficCost per completed taskCost divided by successful outcomesThe only honest efficiency number, since cheap failures are not savings95th percentile versus average costCompare the twoA big gap means a few runaway sessions are inflating the billInput versus output shareInput tokens cost against output costTells you whether the problem is context or verbosityCost by feature and by stepGroup the ledgerFinds the one place most of the money goesMargin by tenantRevenue minus model costFinds customers who cost more than they payPremium-model share on simple stepsSpend on large models for classification, extraction, routingShows routing savingsRetry and duplicate rateRepeated calls divided by total callsPure waste, and usually cheap to fixCache hit shareCached input tokens divided by input tokensShows whether caching actually worksReconciliation gapLedger total minus invoiceProves whether the ledger can be trusted

The most useful calculation borrows from classic accounting: the variance bridge. When the bill rises, split the increase into four causes:

Volume: more sessions than last month.
Price: the provider changed prices.
Mix: traffic moved toward more expensive features or models.
Behavior: the same work now uses more tokens, through longer context, extra steps, or retries.
Only per-call data can produce this split. The provider invoice cannot, and it matters because each cause needs a different response. Volume is good news, price is a negotiation, mix is a product decision, and behavior is an engineering bug.

Why ZizkaDB is the right tool for this job
Plenty of tools can show a usage chart. The accountant’s job needs something different: the ability to go from a total to its cause and defend it. ZizkaDB is built for that.

Every number links back to its cause. Each call is recorded as an event tied to the session and the step that caused it. A cost head is not just a figure. You can click from it to the sessions behind it, replay one, and ask why a particular step happened.

The audit trail doubles as the ledger. Events carry tamper-evident checksums, so the record behind a cost number can be shown to a finance team, a customer, or an auditor as unchanged since it was written. A spreadsheet export cannot offer that.

It sees behavior, which is where cost comes from. The drift view shows when the number of model calls per session or the share of tool steps changes. That is the behavior component of the variance bridge, spotted as it happens instead of at month end.

Your cost data stays in your hands. The open source version is self-hostable, so prompts, usage, and customer identifiers need not go to a third party. For cost data that reveals your margins and your customers’ behavior, that matters.

Customer data can be removed cleanly. If a customer asks for deletion, an erasure function covers both event logs and vector stores, which means tenant-level cost records need not conflict with data protection duties.

It was built for questions you will be asked later. The same record that explains a cost spike also answers what the agent did, why, and what it looked like at the time. One trail serves the accountant, the engineer, and the auditor.

To be clear about limits: ZizkaDB is the ledger and the evidence, not an accounting package. The rate card, the reconciliation, and the judgment stay with the accountant. And I could not confirm a built-in cost report, so expect to assemble the aggregation yourself at first, from the token fields you log.

What goes into the ledger is small. One event per model call, with fields like these:

python

illustrative ledger entry data, one per LLM call

{"model": "model-large", "feature": "support_reply", "step": "retrieval_answer",
"tenant": "tenant_a", "prompt_version": "v14",
"input_tokens": 9200, "output_tokens": 410, "cached_input_tokens": 0}
Store tokens, not money, and apply a dated rate card when you analyze, so history stays correct when prices change.

Worked example: dissecting one invoice
This example is modeled, not measured. It shows what the work produces once the ledger exists. Take a product whose provider invoice for October says one line: 12,000 euro.

Head 1: by feature
FeatureCostShareSupport reply5,40045%Document summarization3,00025%Ticket classification and routing1,80015%Voice assistant1,20010%Internal copilot6005%Total12,000100%

First finding: support reply is nearly half the bill. Start there.

Head 2: by step inside support reply
StepCostPlanning and tool selection1,100Retrieval and context assembly2,300Final answer1,500Retries and validation loops500Total5,400

Second finding: the answer itself is under a third of the feature’s cost. Retrieval and context assembly is the largest part, and retries cost 500 euro for work that produced nothing.

Head 3: by model, and input versus output
CutCostShareLarge model8,40070%Small model3,60030%Input tokens7,80065%Output tokens4,20035%

Third finding: input dominates, so this is a context problem more than a verbosity problem. And all 1,800 euro of ticket classification runs on the large model, a task a small model usually handles.

Head 4: by tenant, with margin
TenantModel costMonthly revenueMarginTenant A2,9002,000–900Tenant B1,7004,0002,300Other 38 tenants7,400not shownaverage cost about 195 eachTotal cost12,000

Fourth finding: Tenant A is a quarter of the bill and loses money. That is a pricing or usage-limit conversation, and it was invisible on the invoice.

Head 5: the excess, meaning cost that bought nothing
Excess headCostPremium model on simple steps (excess over a small model)1,500Context bloat (history and documents resent unnecessarily)1,300Retry loops700Duplicate calls400Total excess3,900

That is 32.5 percent of the invoice. Recovering half to two thirds of it is a realistic goal, which is roughly 1,950 to 2,600 euro a month.

The variance bridge
Last month’s bill was 9,500 euro. This month’s is 12,000. Why?

CauseChangeVolume (more sessions)+1,100Price (provider adjustment)-300Mix (more summarization and voice)+600Behavior (more tokens per session)+1,100Total change+2,500

Without the ledger, the answer to why did it rise would have been traffic grew. With it, you can say that 44 percent of the increase was behavior, which is fixable, and then find which prompt change caused it.

Drilling into one number
The accountant sorts support sessions by cost. The average session costs 0.90 euro, but one costs 41 euro. Opening it in session replay shows 23 model calls. Asking why on the first expensive step shows the chain: an order lookup tool timed out, the agent retried, timed out again, and each retry resent a 14,000 token history. The cause is not the model. It is a slow tool with no retry cap. The fix is a retry limit and a trimmed history, and the saving applies to every session that hits the same pattern.

That is the work: from 12,000 euro, to a feature, to a step, to a session, to a cause, to a fix. The ability to follow that path is what the role is paid for.

What it adds up to
In this example, the first close produces four decisions: cap retries, route classification to a smaller model, trim retrieval context, and open a pricing conversation with Tenant A. Together they target the 3,900 euro of excess and the 900 euro monthly loss on one account. The labor is a few engineer-days to build the ledger and a small project per fix, so payback is typically a few months, with the saving recurring. Treat these as modeled figures and measure your own before promising anything.

Common mistakes to avoid
Skipping reconciliation. A ledger that never meets the invoice is an opinion.

Storing money instead of tokens. Prices change, so store tokens and apply a dated rate card.

Judging by cost per call. Cost per completed task is the number that matters.

Cutting cost without watching quality. A cheaper model that causes rework raises total cost.

Reporting without recommending. A variance with no proposed fix gets ignored.

Logging full prompts everywhere. It adds volume and risk, and cost accounting does not need it.

Start this week
You do not need a department. Pick one product feature, record every model call for it with model, tokens, step, and tenant, keep a dated price table, and compare your total with the invoice at month end. When the two numbers meet, you have a ledger. When you can explain a variance, you are doing the job.

If you tell me your provider, your models, and your rough monthly spend by feature, I can help draft your chart of accounts and a first close template.

Want to test ZizkaDB as your ledger? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB Python SDK wiki · ZizkaDB connection guide · ZizkaDB repository README