五階段 Token Ledger 教學：計算 LLM 每個合格輸出的 token 消耗

Every free-tier LLM project I look at has the same blind spot. The dashboard says requests, the bill says zero, and nobody can answer the one question that matters: how many tokens did we burn to get one output we actually kept?

So here is the conclusion up front. Tokens per call is a vanity metric. Tokens per accepted output is the number that decides whether free inference is cheap or just deferred cost. In this tutorial we build the ledger that produces that number — from an empty file to a report you can paste into a design review.

Five stages. Each one has code and a verification step, so you never advance on a stage you haven't seen work. The code is provider-agnostic on purpose: it talks to any chat endpoint behind a thin adapter, so the ledger stays useful no matter which backend you point it at.

One practical note on where to run it. I use MonkeyCode's free model access behind the adapter in Stage 2 so the meter has something real to measure without a card on file, and the free server option to keep the tiny stub endpoint alive while I iterate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Both of those are availability claims supplied by the project, not benchmarks — quotas, limits, and terms change, so read the current ones on the site before you plan a sprint around them. The ledger below works the same either way.

Stage 1 — Define the receipt before you write the client

Why start with the data shape? Because every metering project I've watched die, died the same way: somebody added a field in week three and the historical numbers stopped being comparable.

Freeze the receipt first. Keep it boring and JSON-serializable.

# ledger/receipt.py
from dataclasses import dataclass, asdict, field
import json, time, uuid

@dataclass
class Receipt:
    request_id: str = field(default_factory=lambda: uuid.uuid4().hex[:12])
    stage: str = "draft"          # which pipeline step asked
    backend: str = "free"         # "free" | "paid" — label it, never infer it
    model: str = "unknown"
    prompt_chars: int = 0
    output_chars: int = 0
    usage_prompt: int | None = None    # provider-reported, None when absent
    usage_output: int | None = None
    latency_ms: int = 0
    status: str = "ok"                # ok | error | truncated
    accepted: bool | None = None      # None = not judged yet

    def to_json(self) -> str:
        return json.dumps(asdict(self), sort_keys=True)

Two fields carry all the weight. backend because a free run and a paid run are not the same observation and must never be averaged together. accepted because a receipt with no verdict is just a timestamp with extra steps.

Verify:

python -c "from ledger.receipt import Receipt; r=Receipt(); print(r.to_json()); assert r.accepted is None"

If that assertion fails, you've already started guessing. Stop and fix it.

Stage 2 — Meter the call, and label every estimate

Now the adapter. The trap here is pretending you know the token count. You often don't — many endpoints return a usage block, some don't, and the ones that do disagree about what counts.

So: prefer provider numbers, fall back to an estimate, and mark the estimate as an estimate.

# ledger/meter.py
import time
from .receipt import Receipt

def estimate_tokens(text: str) -> int:
    # rough, deliberately conservative; NOT a tokenizer
    return max(1, len(text) // 4)

def metered_call(adapter, prompt: str, stage: str, model: str) -> tuple[str, Receipt]:
    rec = Receipt(stage=stage, model=model, backend=adapter.backend,
                  prompt_chars=len(prompt))
    t0 = time.monotonic()
    try:
        out, usage = adapter.chat(prompt)   # usage: dict | None
        rec.status = "ok"
    except Exception:
        rec.status = "error"
        rec.latency_ms = int((time.monotonic() - t0) * 1000)
        return "", rec
    rec.latency_ms = int((time.monotonic() - t0) * 1000)
    rec.output_chars = len(out)
    if usage:
        rec.usage_prompt = usage.get("prompt_tokens")
        rec.usage_output = usage.get("completion_tokens")
    else:
        rec.usage_prompt = estimate_tokens(prompt)
        rec.usage_output = estimate_tokens(out)
    return out, rec

The adapter.chat() signature is a stub by design — write your own transport for whatever endpoint you're testing. Mine is twenty lines and returns (text, usage_or_none). That's the whole contract.

Verify with a fake, before you spend a single token:

# tests/test_meter.py
from ledger.meter import metered_call

class FakeAdapter:
    backend = "free"
    def chat(self, prompt):
        return "ok", {"prompt_tokens": 11, "completion_tokens": 2}

class NoUsageAdapter(FakeAdapter):
    def chat(self, prompt):
        return "ok", None

def test_provider_numbers_win():
    _, rec = metered_call(FakeAdapter(), "hello there", "draft", "m")
    assert (rec.usage_prompt, rec.usage_output) == (11, 2)

def test_estimate_is_still_recorded():
    _, rec = metered_call(NoUsageAdapter(), "hello there", "draft", "m")
    assert rec.usage_prompt > 0 and rec.status == "ok"

python -m pytest tests/test_meter.py -q

Green? Now point the adapter at a real free endpoint and run ten prompts. You should see usage_prompt populated or your estimate filling in — either way, no receipts with zeros.

Stage 3 — Put a gate in front of the meter

Metering after the fact is accounting. Metering before the call is control. The gate answers: is the next request even allowed?

Two rules earn their keep immediately. A projected-token ceiling, and an error-rate circuit breaker — because a free endpoint that starts 500-ing is burning your retry budget, not your token budget.

# ledger/gate.py
from collections import deque

class Gate:
    def __init__(self, max_tokens: int, window: int = 20, max_error_rate: float = 0.25):
        self.max_tokens = max_tokens
        self.window = deque(maxlen=window)
        self.max_error_rate = max_error_rate
        self.spent = 0

    def record(self, rec):
        if rec.status == "error":
            self.window.append(1)
        else:
            self.window.append(0)
            self.spent += (rec.usage_prompt or 0) + (rec.usage_output or 0)

    def allow(self, projected_prompt_tokens: int) -> tuple[bool, str]:
        if self.spent + projected_prompt_tokens > self.max_tokens:
            return False, "ceiling"
        if self.window and sum(self.window) / len(self.window) > self.max_error_rate:
            return False, "error_rate"
        return True, "ok"

Verify with a table of cases, not vibes:

from ledger.gate import Gate
from ledger.receipt import Receipt

def test_gate_stops_at_ceiling():
    g = Gate(max_tokens=100)
    g.record(Receipt(usage_prompt=60, usage_output=30))
    assert g.allow(20) == (False, "ceiling")
    assert g.allow(5)[0] is True

def test_gate_trips_on_errors():
    g = Gate(max_tokens=10_000, window=4, max_error_rate=0.25)
    for _ in range(2):
        g.record(Receipt(status="error"))
    assert g.allow(10) == (False, "error_rate")

The ceiling number is yours to pick — start smaller than you think is polite. The point is that the gate refuses before the request leaves your process, so a surprise quota change produces a clean stop, not a half-finished batch.

Stage 4 — Record acceptance, or the ledger lies

Here's where most metering tutorials quietly cheat. They count tokens and stop. But an output that fails your parser cost you the same tokens as one that passed.

So attach a verdict. Not a human opinion — a mechanical check: does it parse, does the test pass, does the field exist?

# ledger/accept.py
import json

def judge_json(rec, text: str) -> bool:
    try:
        json.loads(text)
        rec.accepted = True
    except json.JSONDecodeError:
        rec.accepted = False
    return rec.accepted

Verify both directions:

from ledger.receipt import Receipt
from ledger.accept import judge_json

def test_accepts_valid():
    r = Receipt()
    assert judge_json(r, '{"a": 1}') is True

def test_rejects_garbage():
    r = Receipt()
    assert judge_json(r, 'Sure! Here is your JSON:') is False

That second test is the one people skip, and it's the one that shows up in production. A model that answers conversationally instead of emitting JSON hasn't failed loudly — it has failed expensively.

Stage 5 — Print one number: tokens per accepted output

Now the payoff. Read your receipts back, split by backend, and compute the ratio.

# ledger/report.py
import json
from collections import defaultdict

def report(path: str) -> dict:
    buckets = defaultdict(lambda: {"tokens": 0, "accepted": 0, "calls": 0, "errors": 0})
    with open(path) as f:
        for line in f:
            r = json.loads(line)
            b = buckets[r["backend"]]
            b["calls"] += 1
            b["tokens"] += (r["usage_prompt"] or 0) + (r["usage_output"] or 0)
            b["errors"] += r["status"] == "error"
            b["accepted"] += bool(r["accepted"])
    for name, b in buckets.items():
        b["tokens_per_accepted"] = round(b["tokens"] / b["accepted"], 1) if b["accepted"] else None
    return dict(buckets)

python -c "from ledger.report import report; import json; print(json.dumps(report('runs/2026-10-26.jsonl'), indent=2))"

Verify the report against a hand-built fixture first. Two receipts, one accepted, 100 tokens total, ratio must be 100.0. If your fixture disagrees with the function, the function is wrong — not the fixture.

The decision table

What do you do with the ratio? Compare it honestly.

That third row is the whole article. A free endpoint at 3,000 tokens per accepted output can lose to a paid one at 400.

Limitations, and who should skip this

The estimator is not a tokenizer. len(text) // 4 is a blunt instrument and will mislead you on non-English text, code, or anything with heavy punctuation. Treat estimated numbers as directional only.

Provider usage blocks are not standardized. If yours is missing fields, your ledger will silently under-count — add an assertion, not a comment.

Free availability is not an SLA. Both claims I mentioned earlier are claims, not guarantees; the gate exists precisely because the ground can move under you.

And the acceptance check is the weakest link. json.loads is easy. A semantic grader is not. If you can't define "accepted" mechanically, you can't compute the ratio — build that first.

Skip this approach if you're in a regulated path needing audit-grade billing, if your latency budget is under a second end-to-end, or if your volume is high enough that the free tier is a rounding error anyway.

Close

Five stages, maybe ninety minutes, and you end up with a number that survives contact with a design review: tokens per accepted output, split by backend, with the errors counted instead of hidden.

If standing up the endpoint is the part that keeps holding you back, MonkeyCode's free server option is one way to get a live target for Stage 2 — verify the current terms yourself and let the ledger do the arguing.