ReceiptRack：在本機整理收據，不把收據傳到其他地方

Every freelancer I know has the same shoebox: a folder of receipt photos and invoice PDFs that gets sorted once a year, in a panic, before taxes. Cloud OCR APIs exist for this — and for exactly one reason (my clients' financial data never leaving my laptop, and paying $0 instead of per-page forever), I built ReceiptRack on a local open-weight model instead.

Drop receipts in → a vision model running on your machine transcribes and structures them → suspicious extractions get flagged for a 10-second human review → keyword rules file them to the right client → export a CSV your accountant accepts.

flowchart LR
    A[Receipt / invoice<br/>PNG · JPG · PDF] --> B[Preprocess<br/>Pillow]
    B --> C[Vision pass<br/>gemma3:4b via Ollama<br/>faithful transcription]
    C --> D[Structuring pass<br/>schema-constrained JSON<br/>temperature 0]
    D --> E{Validator<br/>flags · sum check}
    E --> F[(SQLite, local)]
    F --> G[Human review UI]
    G --> H[Client sorting<br/>rules + model]
    H --> I[CSV export]

Why open is the business model, not a preference

The open pieces are the product: gemma3:4b (open-weight vision model), Ollama (local runtime), FastAPI + SQLite, MIT-licensed. A closed API structurally cannot promise "this runs on your laptop" — the privacy claim is only true because of what's open.

What the community had already figured out (and what it hadn't)

Two DEV posts shaped the design:

• Bytefer's Ollama-OCR for High-Precision OCR with Ollama established the local vision-OCR pipeline — including a receipt demo. But the most interesting part is what the comments didn't resolve: "Does it hallucinate with large tables with lot of numbers?" went unanswered except by "go test the 90B demo yourself."
• Ajit Kumar's Complete Guide to Ollama makes the economics case plainly: "Your data never leaves your machine… no API costs."

And from an open-source invoice extractor's published eval: invoice # 20%, date 60%, total 60% field accuracy with a small vision model. Read that before you ship anything that auto-files money documents. So ReceiptRack's core rule became: the model proposes, a human disposes. Every extraction lands in a review queue with flags:

• missing_total, missing_vendor, date_not_iso
• line_items_mismatch — line items (+ tax) don't sum to the stated total

That last flag caught a real case in testing: an Uber receipt where the $4 tip isn't a line item. A human glances at it; the model never silently files a wrong number.

The five things that broke, and the fixes that stuck

1. Two passes beat one. First version: image → JSON. Final: image → faithful transcription → JSON from the transcription. Same accuracy ceiling, but you get an auditable raw transcript in the UI, and re-parsing doesn't re-run the vision model.

2. Optional schema fields = missing data. The model happily returned vendor: "" and skipped category entirely when they had defaults. Making them required in the JSON schema — with Ollama's schema-constrained decoding — turned "usually present" into grammatically impossible to omit.

3. A prompt that returns null for the total, deterministically. The docs clearly said TOTAL DUE 65.69; three runs in a row, total: null. The fix was one unambiguous line:

- total: the grand total the customer pays (TOTAL, TOTAL DUE, Amount Due).
  If the document says "TOTAL DUE 65.69", then total = 65.69.
  Never null when a total line exists.

Temperature 0 doesn't save a confusing prompt — it just makes the confusion reproducible.

4. The model echoing the vendor as the client name. Auto-sorting asked "which client should this go to?" and gemma3:4b kept answering "Figma Inc." (the vendor) no matter how sternly the prompt said "only names from the list." Instructions lost. So I stopped asking nicely and constrained the grammar:

schema = {
    "type": "object",
    "properties": {
        "client": {
            "anyOf": [
                {"type": "string", "enum": [c["name"] for c in clients]},
                {"type": "null"},
            ]
        }
    },
    "required": ["client"],
}

Wrong answers became unrepresentable. This is the single best argument for open, schema-constrained local inference I have: a closed chat API gave me a JSON "format" hint; Ollama gave me a grammar.

5. Malformed JSON still happens. Even under schema constraints, one extraction in three crashed the parser. Retries (strict schema → loose JSON → validate) turned a hard failure into noise.

The numbers

• ~5–15 s per receipt on an M-series Mac, 100% GPU, 16 GB RAM
• 3/3 correct vendor/date/total/tax/line items on the test set after fix #3
• Cost per receipt: $0.00 — forever

Honest limitations

Small open-weight models misread faded thermal prints — that's what the review queue is for. The LLM client-assignment fallback makes defensible-but-imperfect choices (a 4B model thinks every travel expense belongs to the design agency); deterministic keyword rules run first for that reason. Single-user, localhost-only by design.

Try it

ollama pull gemma3:4b
git clone https://github.com/kondekarshubham123/ReceiptRack.git && cd receipt-rack
uv sync
uv run uvicorn app.main:app --port 8300   # → http://127.0.0.1:8300
uv run python scripts/make_samples.py     # demo receipts

Repo: https://github.com/kondekarshubham123/ReceiptRack · MIT · README has the full architecture and the env vars (RECEIPT_RACK_MODEL, OLLAMA_URL).

Sources:

🌐 Community Wisdom: [Ollama-OCR for High-Precision OCR with Ollama]

Source: [Bytefer]
Tags: webdev, programming, javascript, ai

Establishes the local vision-model OCR pipeline as a proven pattern; the comments' unresolved question about hallucinated numbers in tables is why this project validates arithmetic instead of trusting the model.

🔗 Read Full Discussion

🌐 Community Wisdom: [The Complete Guide to Ollama: Run Large Language Models Locally]

Source: [Ajit Kumar]
Tags: ollama, llm, rag, ai

The privacy/no-API-cost case this project's pitch rests on — with the honest tradeoff surfaced by readers: hardware is the real cost of going local.

🔗 Read Full Discussion