跳到正文
DEV Community· Kamal Mosadegh·· 5 小時前AI 評分56

pdf-slice 以快取校驗及 OCR 回退處理 PDF 擷取靜默錯誤

PDF Extraction Kept Failing Silently — So I Built an Extractor That Refuses to Be Quietly Wrong

AI 導讀

作者 Kamal Mosadegh 開發的 pdf-slice 是一款本機、僅用 CPU 的 PDF 轉 Markdown 工具,會在輸出可能錯誤或不完整時發出警告,而非靜默返回內容。

正文

Every PDF pipeline I have used shares the same bad habit: when extraction goes wrong, it goes wrong quietly. You get a 200 status, some markdown, and no idea that half the document is missing — or that it is the wrong document entirely.

pdf-slice is my attempt at the opposite contract: a small, local, CPU-only PDF-to-markdown tool that never silently returns wrong content. If the output might be wrong or incomplete, it says so, loudly.

Failure mode 1: the cache served a different document

Several different files shared the name Nexus.pdf — and one cache key. A user asking for the April report silently received the March report instead, with correct-looking metadata. The fix is boring and absolute: pdf-slice keys its cache by full URL + SHA-256 of the downloaded bytes, never the filename, and re-verifies the content hash before serving anything cached. Mismatch means fresh extraction, never the wrong document.

Failure mode 2: the text was there, but invisible

A 46-page PDF with a full text layer came back as ~3,000 header-only characters. The body text was white, nested inside form XObjects under an opaque image — and the classifier had confidently declared "text-based, no OCR needed". Other extractors found 86,000+ characters in the same file.

pdf-slice treats a suspiciously thin native extraction as a signal, not an answer: it falls back to OCR automatically, and always reports which path produced the output.

The design rules

  1. Identity is a hash, not a name. Two files with the same name and different hashes are different documents.
  2. Thin output is a signal. Below a per-page threshold, OCR runs; if it is still thin afterwards, the result is labelled low-confidence (fallback) with a warning.
  3. Every page carries provenance. The JSON metadata records each page's origin (native/OCR), character count, tables and timing — plus the document's content_hash and cache state.
  4. Fail loudly. Encrypted, corrupt, empty or non-PDF inputs produce a clear error — never empty "successful" output.

Outputs: clean markdown (multi-column reading order, tables as markdown tables), provenance JSON, and RAG chunks in JSONL — each chunk with text, page, element_type, confidence, source (URI + content hash) and reading_order.

Benchmark

On the fixtures bundled with the tool (generated by its own test suite):

File Path Word recovery Seconds/page Silent errors
white native 100% 1.575 0
scanned ocr 100% 2.802 0
multilingual native 100% 1.401 0
table native 100% 1.632 0
large (120 pages) native — 0.462 0

Total silent errors: 0 — verified by a 26-group test matrix (cache collisions, white text, scanned, multilingual, tables, columns, corrupt, encrypted, batch mode, MCP, clean installs), each test repeated 3×.

MCP server included

pdf-slice ships an MCP server (stdio) for Claude Code, Cursor and Codex, with two tools: extract_pdf and verify_pdf — the latter extracts the same document from two routes and reports a match_rate, so a route serving different content is caught instead of trusted. There is also a ready-made agent skill in the repo.

Try it

git clone https://github.com/kamalmosadegh/pdf-slice && cd pdf-slice
python -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/python extract.py your-file.pdf

If you feed PDFs into LLM pipelines or RAG systems, I would genuinely like to hear where this breaks for you — issues and feedback very welcome.

來源:DEV Community · dev.to