跳到正文
LlamaIndex:產品、工程與評測·· 3 小時前AI 評分54

LlamaIndex 發佈文件解析模型 OCRia-1

Introducing OCRia-1

AI 導讀

LlamaIndex 發佈 OCRia-1,這是一個參數量為 4 billion 的視覺語言模型,在 ParseBench 的整體文件解析評分為 87.4,較次高模型高 12.1 分。

正文

Today, we are introducing OCRia-1, the world’s best document parsing model. It is a 4-billion-parameter vision-language model (VLM) that is state-of-the-art in both accuracy, cost and latency.

On ParseBench, OCRia-1 sets a new high score for any parsing model under 35 billion parameters: 87.4 overall across text content, formatting, tables, and charts, 12.1 points ahead of the next-highest parsing model. It also leads table evaluations across three third-party benchmarks: olmOCR-bench* (92.9), PulseBench-Tab (84.1), and fr-bench-pdf2md.

OCRia-1 4B87.4

LightOnOCR-3-4B75.3

Chandra-OCR-2 (4B)74.8

Mistral OCR 4.167.5

LightOnOCR-2-1B61.1

Surya OCR 2 (0.7B)58.4

dots.mocr (3B)55.6

olmOCR-2 (7B)43.3

That matters because demand for document parsing is growing exponentially with the rise of agentic automation in professional work. Most of the information in enterprise documents, such as financial reports, insurance filings and tax forms, is organized as tables, charts, diagrams and formatted text. VLMs have become the standard for digitizing these documents, yet both frontier models and existing open-weight models still struggle to parse them accurately, cheaply and quickly at scale. OCRia-1 is the best model available today at all three.

State-of-the-art document parsing

OCRia-1 achieves the highest overall scores among the systems we evaluated on ParseBench, PulseBench-Tab, and fr-bench-pdf2md, with a model of just 4 billion parameters. On ParseBench, it scores 87.4, a 12.1-point lead over the next-highest system.

The evaluations span different languages, document types, and scoring methods from complete document pages to multilingual tables.

Overall document parsing results (we compare all systems at the same rendering for a fair comparison keeping benchmarks own standard rendering rules. *Scores are adjusted for olmOCR-bench based on standard 1,288px rendering for all models).

ParseBench

OCRia-1 4B87.4

LightOnOCR-3-4B75.3

Chandra-OCR-2 (4B)74.8

dots.mocr (3B)55.6

olmOCR-2 (7B)43.3

olmOCR-bench

OCRia-1 4B83.5

olmOCR-2 (7B)82.3

LightOnOCR-3-4B81.2

Chandra-OCR-2 (4B)81.2

dots.mocr (3B)80.1

PulseBench-Tab

OCRia-1 4B84.1

LightOnOCR-3-4B82.4

dots.mocr (3B)82.4

Chandra-OCR-2 (4B)80.0

olmOCR-2 (7B)79.5

fr-bench-pdf2md

OCRia-1 4B72.1

LightOnOCR-3-4B71.4

Chandra-OCR-2 (4B)65.5

olmOCR-2 (7B)56.9

dots.mocr (3B)56.8

Reconstructing complex tables

Complex tables are a demanding test of document understanding. A parser must recover nested headers, merged cells, and irregular grids while preserving the relationships between them. Losing those relationships can turn an accurate transcription into an incorrect financial comparison or a misinterpreted coverage limit.

We made this a central focus of OCRia-1. It reconstructs nested headers, merged cells, and irregular grids, preserving the relationships between values and their labels.

OCRia-1’s table accuracy makes it especially well suited to financial services and insurance, where interpreting financial statements and coverage schedules requires preserving complex table structure.

OCRia-1 leads table-specific evaluations across all four benchmarks we tested, including three developed independently. Its lead spans ParseBench, olmOCR-bench*, PulseBench-Tab’s multilingual tables, and both long-table categories in fr-bench-pdf2md.

ParseBenchTables dimension

OCRia-1 4B91.5

Chandra-OCR-2 (4B)89.2

dots.mocr (3B)85.2

LightOnOCR-3-4B82.5

olmOCR-2 (7B)73.0

olmOCR-benchTables category

OCRia-1 4B92.9

Chandra-OCR-2 (4B)92.1

dots.mocr (3B)90.7

LightOnOCR-3-4B90.2

olmOCR-2 (7B)84.3

PulseBench-TabAll 1,820 tables

OCRia-1 4B84.1

LightOnOCR-3-4B82.4

dots.mocr (3B)82.4

Chandra-OCR-2 (4B)80.0

olmOCR-2 (7B)79.5

fr-bench-pdf2mdLong table

OCRia-1 4B83.2

LightOnOCR-3-4B80.6

Chandra-OCR-2 (4B)80.6

olmOCR-2 (7B)67.9

dots.mocr (3B)56.7

fr-bench-pdf2mdLong table 2

OCRia-1 4B81.9

Chandra-OCR-2 (4B)77.8

LightOnOCR-3-4B70.7

olmOCR-2 (7B)63.9

dots.mocr (3B)59.8

These benchmarks use different datasets and scoring methods, testing performance across multilingual tables, complex structures, and long tables. These results establish complex tables as a defining strength of OCRia-1 across different datasets and evaluation methods.

Turning charts into structured data

Real-world charts come in many forms: stacked bars, overlapping lines, multiple axes, and data points without explicit value labels. Recovering their data means reading the scales, distinguishing the series, and matching each value to the right label. A summary like “revenue grew across all segments” isn’t enough for calculations or analysis—you need the actual numbers. OCRia-1 converts charts into HTML tables of their data points, keeping values associated with the correct series and labels.

ParseBench evaluates both the numbers and those relationships: a value receives credit only when it appears under the correct row and column headers.On ParseBench’s chart evaluation, OCRia-1 scores 85.1—a 19.0-point lead over the next-closest model (66.1), while conventional parsers score near zero

OCRia-1 4B85.1

LightOnOCR-3-4B66.1

Chandra-OCR-2 (4B)65.1

Mistral OCR 4.140.1

LightOnOCR-2-1B15.2

olmOCR-2 (7B)13.7

Surya OCR 2 (0.7B)1.2

dots.mocr (3B)0.1

Preserving formatting and mathematical notation

Most parsers strip inline styling, dropping superscripts, subscripts, strikethroughs, and form checkboxes. OCRia-1 retains these marks in Markdown and HTML, and transcribes mathematical formulas into LaTeX rather than plain text. On ParseBench’s formatting dimension, it scores 83.4, compared to 67.9 for the next-highest system.

OCRia-1 4B83.4

Surya OCR 2 (0.7B)67.9

Mistral OCR 4.166.4

LightOnOCR-3-4B63.5

LightOnOCR-2-1B63.5

Chandra-OCR-2 (4B)61.4

dots.mocr (3B)47.0

olmOCR-2 (7B)2.2

Data and Training

The magic of OCRia-1 lies in the data and training pipeline. We built a training set of 100k + document pages aimed at the cases modern VLMs still get wrong, from tables and charts that are both dense and span multiple pages to mathematical notation, multilingual text and difficult scans.

We bootstrap ground-truth annotations using generation and review VLMs, together with human annotations. Synthetic document generators supplement the real-world data, allowing us to vary document complexity and diversity while matching the long-tail distribution of real-world documents.

We post-trained OCRia-1 using a combination of supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR). SFT gives the model a base layer of parsing capability. Our RL tasks build on that foundation, using carefully designed rewards to improve the model’s ability to generalize across complex tables and charts.

來源:LlamaIndex:產品、工程與評測 · llamaindex.ai