# Document conversion quality

> mdapi.io - Minimal Data API I/O: a content transformation layer primitive for AI systems.
>
> Transforms files and webpages into AI-ready Markdown and structured data, optimized for LLM efficiency and token usage.

Quality measured over three corpora - PDF documents, those documents re-emitted in fifteen office, web, spreadsheet and Apple formats, office files and e-books, and a sample of iWork packages, archives, data and mail - each scored by its own evaluator: the scores of this service and of the other engines on the same files, which corpus measures each extension, who measured each, the method boundaries, and the capabilities no compared engine offers.

200 PDF documents, those same 200 documents written out in fifteen office, web, spreadsheet and Apple formats, 63 office files and e-books, and 26 samples of iWork packages, archives, data and mail, each family scored by its own evaluator over its own corpus.

Each row is the answer that engine gives by default. The `mdapi` rows are the native path: the file is read by the reader itself, text only, with no model in the path, so the figure is the answer a caller gets at no model cost. Every other row is its engine's own default pipeline, and that pipeline may run a model of its own - docling, marker, mineru, pymupdf4llm 2.1 and the hybrid opendataloader configuration run layout and table models inside it, so their figures carry that model's work and its time as well as the reader's. The comparison is between what each tool returns by default, not between two readers of one kind. A prompt on top of any of them runs a model over the converted text and can recover what a layout-only reader leaves out - a table whose columns do not line up, a heading whose level the font does not state - at a longer answer time and a higher cost than these figures measure.

## The whole picture

- **iWork, archives, data and mail** - anydoc bench coverage manifest, 26 documents. Metric: documents produced; `mdapi`: 24 of 24.
- **Office formats and e-books** - anydoc bench corpus, 63 documents. Metric: recall of ground-truth units; `mdapi`: 0.9886 over 40 documents.
- **PDF** - opendataloader-bench, 200 documents. Metric: overall score; `mdapi`: 0.9031.
- **The same documents in fifteen formats** - opendataloader-bench, re-emitted in fifteen formats, 200 documents. Metric: overall score; `mdapi`: 0.9337 to 0.9920 across every format.

## All results in one table

Every corpus on this page in one table, one row per document set. The first row is opendataloader-bench - 200 PDF documents. The rows after it are the same 200 documents written out in each format of the `formats` corpus. After those come the files of the `office` corpus, one row per format, and then the samples of the `reach` corpus, one row per extension. A cell is that engine's number on that set: the evaluator's overall score for `PDF` and `formats`, recall of ground-truth units for `office`, and documents produced for `reach` out of the samples that carry content. The three are different scales and no figure of one is averaged with another; a column reads one engine down all four corpora, a row compares engines on one set.

| Corpus and set           | `mdapi` | anydoc | docling    | LibreOffice | markitdown | mammoth   | pandoc    | unstructured |
| ------------------------ | ------- | ------ | ---------- | ----------- | ---------- | --------- | --------- | ------------ |
| PDF, 200 documents       | 0.9031  | 0.8665 | 0.8873     | 0.670       | 0.5885     | n/r       | n/r       | 0.6874       |
| formats, DOCX 200        | 0.992   | 0.9901 | 0.9899     | 0.9623      | 0.992      | 0.891     | 0.9919    | 0.9924       |
| formats, XLSX 200        | 0.9643  | 0.8061 | 0.6603     | 0.6978      | 0.8511     | n/r       | n/r       | 0.7865       |
| formats, PPTX 200        | 0.9409  | 0.941  | 0.9271     | 0.6299      | 0.8633     | n/r       | n/r       | 0.8646       |
| formats, DOC 200         | 0.9915  | 0.9897 | 0.9848     | 0.9579      | n/r        | n/r       | n/r       | 0.9873       |
| formats, XLS 200         | 0.9747  | 0.8131 | 0.6601     | 0.7104      | 0.8597     | n/r       | n/r       | 0.7865       |
| formats, PPT 200         | 0.9337  | 0.8413 | 0.930      | 0.6298      | n/r        | n/r       | n/r       | 0.8698       |
| formats, ODT 200         | 0.991   | 0.9867 | 0.9897     | 0.9388      | n/r        | n/r       | 0.9741    | 0.979        |
| formats, ODS 200         | 0.9792  | 0.8131 | 0.6598     | 0.7104      | n/r        | n/r       | n/r       | n/r          |
| formats, ODP 200         | 0.941   | 0.9407 | 0.9152     | 0.6299      | n/r        | n/r       | n/r       | n/r          |
| formats, RTF 200         | 0.9651  | 0.9503 | 0.7358     | 0.6887      | 0.5186     | n/r       | 0.965     | 0.9677       |
| formats, HTML 200        | 0.966   | n/r    | 0.9414     | 0.9428      | 0.9432     | n/r       | 0.8934    | 0.9419       |
| formats, EPUB 200        | 0.9904  | 0.8924 | 0.921      | n/r         | 0.8686     | n/r       | 0.8694    | 0.9616       |
| formats, PAGES 200       | 0.9871  | n/r    | 0.991      | 0.6185      | n/r        | n/r       | n/r       | n/r          |
| formats, NUMBERS 200     | 0.9654  | n/r    | 0.662      | 0.6318      | n/r        | n/r       | n/r       | n/r          |
| formats, KEY 200         | 0.9871  | n/r    | 0.7404     | 0.569       | n/r        | n/r       | n/r       | n/r          |
| office, docx 11          | 1.0     | 0.998  | 0.902 9/11 | 0.995 10/11 | 0.995      | 0.992     | 0.920     | 0.997 9/11   |
| office, xlsx 2           | 0.931   | 0.879  | 0.948      | 0.879       | 0.983      | n/r       | n/r       | 0.983        |
| office, xls 1            | 0.931   | 0.793  | f          | 0.759       | 0.862      | n/r       | n/r       | 0.862        |
| office, pptx 7           | 1.0     | 1.0    | 1.0 1/7    | 0.600 4/7   | 1.0 2/7    | n/r       | n/r       | 0.967 3/7    |
| office, odt 6            | 1.0     | 1.0    | f          | 0.990 3/6   | n/r        | n/r       | 0.990 3/6 | 1.0 3/6      |
| office, ods 2            | 1.0     | 1.0    | f          | 1.0 1/2     | n/r        | n/r       | n/r       | n/r          |
| office, odp 1            | 1.0     | 1.0    | f          | 0.923       | n/r        | n/r       | n/r       | n/r          |
| office, csv 4            | 1.0     | 1.0    | 1.0        | f           | 1.0        | n/r       | 0.667 3/4 | 1.0 3/4      |
| office, epub 4           | 0.938   | 0.829  | f          | n/r         | 1.0        | n/r       | 0.832     | 0.832        |
| office, malformed docx 2 | 1.0     | 0.975  | f          | 0.950 1/2   | 0.950 1/2  | 0.950 1/2 | 0.950     | f            |
| reach, pages 1           | 1/1     | 0/1    | 1/1        | 1/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, numbers 1         | 1/1     | 0/1    | 1/1        | 1/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, key 2             | 2/2     | 0/2    | 2/2        | 2/2         | 0/2        | 0/2       | 0/2       | 0/2          |
| reach, doc 2             | 2/2     | 2/2    | 1/2        | 1/2         | 0/2        | 0/2       | 0/2       | 1/2          |
| reach, xls 1             | 1/1     | 1/1    | 1/1        | 1/1         | 1/1        | 0/1       | 0/1       | 1/1          |
| reach, ppt 1             | 1/1     | 1/1    | 1/1        | 1/1         | 0/1        | 0/1       | 0/1       | 1/1          |
| reach, rar 2             | 2/2     | 0/2    | 0/2        | 0/2         | 0/2        | 0/2       | 0/2       | 0/2          |
| reach, 7z 2              | 2/2     | 0/2    | 0/2        | 0/2         | 0/2        | 0/2       | 0/2       | 0/2          |
| reach, gz 1              | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, zip 2             | 2/2     | 0/2    | 1/2        | 0/2         | 2/2        | 0/2       | 0/2       | 0/2          |
| reach, tar 1             | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 1/1       | 0/1       | 0/1          |
| reach, xz 1              | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, bz2 1             | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, zst 1             | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, parquet 1         | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, db 1              | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, dcm 1             | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, pst 1             | 1/1     | 0/1    | 0/1        | 0/1         | 0/1        | 0/1       | 0/1       | 0/1          |
| reach, mbox 1            | 1/1     | 0/1    | 0/1        | 1/1         | 1/1        | 0/1       | 0/1       | 0/1          |

`n/r` - the engine has no reader for the format, by its own refusal. `f` - the engine declares the format, was run, and returned no document. A count beside an `office` figure is the documents the engine was scored on, stated where it is fewer than the set carries; a figure without one covers every document of its set. A `reach` figure is documents produced out of the samples that carry content, and the count in its set label is that number. No cell is left empty: every one of them is a measured number or one of those facts, so a gap is never read as a zero.


## What the overall score is made of

The overall score is a mean of the components below, and each component answers a different question. For opendataloader-bench - 200 PDF documents; derived from DP-Bench and PubLayNet, with a Markdown ground truth for every document - the evaluator scores:

- **NID** - reading order, over the full extracted text.
- **NID-S** - reading order, with tables stripped before comparison.
- **TEDS** - tables, by structure and cell text.
- **MHS** - headings, by position and aligned content blocks.
- **MHS-S** - headings, by topology only.

Time is each run made here as a percentage of the shortest; a row marked `*` carries no time, because it was not run here.

| Engine                  | Overall | NID    | NID-S  | TEDS   | MHS    | MHS-S  | Time    |
| ----------------------- | ------- | ------ | ------ | ------ | ------ | ------ | ------- |
| nutrient 1.4.3          | 0.9111  | 0.9306 | 0.9401 | 0.8511 | 0.8733 | 0.9287 | 614%    |
| `mdapi`                 | 0.9031  | 0.9303 | 0.9331 | 0.8188 | 0.8594 | 0.9258 | 100%    |
| docling 2.145.0         | 0.8873  | 0.9019 | 0.8554 | 0.9248 | 0.830  | 0.9046 | 46315%  |
| opendataloader hybrid   | 0.8771  | 0.9157 | 0.9049 | 0.6756 | 0.8219 | 0.8882 | 19754%  |
| anydoc                  | 0.8665  | 0.9057 | 0.8878 | 0.8144 | 0.7717 | 0.8495 | 497%    |
| pymupdf4llm 2.1         | 0.8664  | 0.9057 | 0.8888 | 0.7728 | 0.782  | 0.8893 | 2405%   |
| marker 1.6.2            | 0.8563  | 0.8798 | 0.8537 | 0.8715 | 0.7885 | 0.9089 | 54000%  |
| edgeparse 0.3.0         | 0.8445  | 0.8864 | 0.8742 | 0.6967 | 0.756  | 0.8523 | 1592%   |
| opendataloader 2.5.11   | 0.8421  | 0.9117 | 0.9159 | 0.4831 | 0.7569 | 0.8368 | 389%    |
| unstructured [hi_res] * | 0.8414  | 0.9038 | 0.8965 | 0.5883 | 0.7486 | 0.8481 | -       |
| mineru 2.7.0            | 0.8366  | 0.858  | 0.8554 | 0.8682 | 0.7613 | 0.8539 | 49415%  |
| marker 2.1.0            | 0.8317  | 0.8682 | 0.8434 | 0.6764 | 0.7679 | 0.8928 | 160046% |
| pymupdf4llm 0.2.0 *     | 0.7316  | 0.8851 | 0.9166 | 0.401  | 0.4122 | 0.4974 | -       |
| unstructured 0.27.36    | 0.6874  | 0.877  | 0.8528 | 0.0    | 0.4027 | 0.5073 | 11251%  |
| LibreOffice 7.4.3.2     | 0.670   | 0.8668 | 0.8425 | 0.0    | 0.372  | 0.6860 | 2392%   |
| markitdown 0.1.8        | 0.5885  | 0.8437 | 0.8378 | 0.2729 | 0.0    | 0.0    | 900%    |
| liteparse 1.2.1         | 0.5768  | 0.8672 | 0.8439 | 0.0    | 0.0    | 0.0    | 13198%  |

For opendataloader-bench, re-emitted in fifteen formats, the same evaluator scores the answer this service gives on each format:

| Format  | NID    | TEDS   | MHS    |
| ------- | ------ | ------ | ------ |
| DOCX    | 0.9900 | 0.9694 | 0.9959 |
| XLSX    | 0.9825 | 0.8986 | 0.9296 |
| PPTX    | 0.9833 | 0.9618 | 0.8129 |
| DOC     | 0.9896 | 0.9694 | 0.9952 |
| XLS     | 0.9846 | 0.8986 | 0.9672 |
| PPT     | 0.9826 | 0.9169 | 0.8006 |
| ODT     | 0.9896 | 0.9618 | 0.9958 |
| ODS     | 0.9854 | 0.9456 | 0.9681 |
| ODP     | 0.9833 | 0.9618 | 0.8133 |
| RTF     | 0.9842 | 0.9692 | 0.9062 |
| HTML    | 0.9850 | 0.9692 | 0.9082 |
| EPUB    | 0.9885 | 0.9692 | 0.9935 |
| PAGES   | 0.9886 | 0.9263 | 0.9940 |
| NUMBERS | 0.9850 | 0.9025 | 0.9306 |
| KEY     | 0.9886 | 0.9263 | 0.9940 |

## iWork, archives, data and mail

The subject is whether an engine produces a document at all, over samples outside the quality corpus. The corpus is anydoc bench coverage manifest - 26 mixed samples; samples from the same project that the quality corpus does not contain, including iWork packages, archives and compressed streams. Corpus and evaluator are published at https://github.com/firecrawl/anydoc.

The evaluator scores each prediction against the ground truth on these metrics, each normalised so that a higher score is closer to the ground truth:

- **Produced** - the engine returned a document that carries text from the file; an unpacked archive is not a document, and a sample with no content of its own is not counted.
- **Not supported** - the engine has no reader for the format, by its own converter list and the message it raised on the sample.
- **Failed** - the engine declares the format but errored or returned no text.

Documents produced per family.

- **iWork packages, '09 generation** - 6 samples, 4 with content. Produced by `mdapi` 4, docling 4, LibreOffice 4; nothing from anydoc, markitdown, pandoc, mammoth, unstructured.
- **legacy office files** - 4 samples. Produced by `mdapi` 4, anydoc 4, markitdown 1, docling 3, LibreOffice 3, unstructured 3; nothing from pandoc, mammoth.
- **archives** - 11 samples. Produced by `mdapi` 11, markitdown 2, docling 1, mammoth 1; nothing from anydoc, LibreOffice, pandoc, unstructured.
- **data and mail** - 5 samples. Produced by `mdapi` 5, markitdown 1, LibreOffice 1; nothing from anydoc, docling, pandoc, mammoth, unstructured.



Totals over the 24 samples that carry content:

| Engine               | Produced | Not supported | Failed |
| -------------------- | -------- | ------------- | ------ |
| `mdapi`              | 24       | 0             | 0      |
| docling 2.145.0      | 8        | 14            | 2      |
| LibreOffice 7.4.3.2  | 8        | 15            | 1      |
| anydoc               | 4        | 20            | 0      |
| markitdown 0.1.8     | 4        | 20            | 0      |
| unstructured 0.27.36 | 3        | 21            | 0      |
| mammoth 1.12.0       | 1        | 23            | 0      |
| pandoc 3.9           | 0        | 24            | 0      |

## Office formats and e-books

The subject is office documents and e-books, by the native readers, images off and no paid stage in the path. The corpus is anydoc bench corpus - 63 office files and e-books, of which 40 carry a written ground truth; the sample set of the anydoc project, with a ground truth written for this corpus rather than produced by any engine under test. Corpus and evaluator are published at https://github.com/firecrawl/anydoc.

The evaluator scores each prediction against the ground truth on these metrics, each normalised so that a higher score is closer to the ground truth:

- **Recall** - share of ground-truth units whose word tokens appear in the output, in order, anywhere; a unit is a table cell, a paragraph, a list item text or a heading text.
- **Order** - share of the units that were found which also appear in ground-truth order.
- **Table** - share of meaningful ground-truth cells present at the same row and column, by value rather than by spelling.
- **Headings** - share of ground-truth headings emitted at exactly the same level.
- **Lists** - share of ground-truth list items emitted with a list marker.

### Quality

A score appears only for a document an engine produced; the bracketed number is how many documents the metric covers.

| Engine               | Documents scored | Recall | Order    | Table       | Headings   | Lists      |
| -------------------- | ---------------- | ------ | -------- | ----------- | ---------- | ---------- |
| `mdapi` (this tree)  | 40               | 0.9886 | 1.0      | 1.0 (12)    | 1.0 (9)    | 1.0 (9)    |
| anydoc               | 40               | 0.9698 | 1.0      | 0.4571 (12) | 0.7778 (9) | 0.9907 (9) |
| markitdown 0.1.8     | 25               | 0.9891 | 1.0      | 0.6404 (7)  | 1.0 (8)    | 0.9833 (5) |
| pandoc 3.9           | 23               | 0.8833 | 1.0 (22) | 0.3667 (5)  | 0.6296 (9) | 0.9762 (7) |
| LibreOffice 7.4.3.2  | 23               | 0.9004 | 1.0 (22) | 0.2622 (10) | 0.3333 (3) | 0.9833 (5) |
| docling 2.145.0      | 16               | 0.9383 | 0.9946   | 0.7189 (5)  | 0.0 (2)    | 0.9583 (2) |
| unstructured 0.27.36 | 25               | 0.9612 | 1.0      | 0.7537 (7)  | 0.9333 (5) | 1.0 (5)    |
| mammoth 1.12.0       | 12               | 0.988  | 1.0      | 0.3704 (3)  | 1.0 (4)    | 0.9722 (3) |

23 of the 63 files carry no ground truth and are scored for no engine: rtf 7, doc 5, ppt 4, docx 4, xlsb 1, odt 1, ods 1. An engine may still have produced an answer for one of them, so its count in a format is smaller than the number of files it read.

### Below full recall

- **a spreadsheet whose cells are dates, durations, exponents and booleans** - recall 0.862. Missing: a midnight datetime is emitted as its date, a duration as a clock reading, an exponent in upper case, a boolean in upper case. Lost content: No - the same values in other spellings, and the table metric accepts all of them.
- **a spreadsheet whose cells carry stored doubles** - recall 0.931. Missing: the last two digits of a stored double, dropped by the number formatter that every spreadsheet and array reader shares to suppress float noise. Lost content: Partly - the value stays within the tolerance the table metric uses.
- **an e-book whose formula carries both a presentation tree and a TeX annotation** - recall 0.750. Missing: the presentation tree is dropped because the annotation is preferred as the machine-usable form. Lost content: Yes - the only real content loss in this corpus, and a contract choice named here rather than averaged into the score.



### Time

Seconds per document, with what each figure covers.

- **`mdapi` (this tree)** - 63 documents, 0 failed attempts, 0.074 s per document. What the number covers: conversion only, the module tree already loaded, as it is inside the Worker isolate.
- **`mdapi` (one-shot command line)** - 63 documents, 0 failed attempts, 0.895 s per document. What the number covers: process start and the import of the reader tree, then the conversion.
- **anydoc** - 59 documents, 4 failed attempts, 0.002 s per document. What the number covers: the tool's own reported conversion time, process start excluded.
- **anydoc (whole process)** - 59 documents, 4 failed attempts, 0.100 s per document. What the number covers: process start, then the conversion.
- **markitdown 0.1.8** - 27 documents, 8 failed attempts, 0.062 s per document. What the number covers: an in-process call, warm.
- **docling 2.145.0** - 16 documents, 14 failed attempts, 0.117 s per document. What the number covers: an in-process call, warm.
- **mammoth 1.12.0** - 12 documents, 5 failed attempts, 0.201 s per document. What the number covers: the conversion inside a Node process, its own timer.
- **unstructured 0.27.36** - 35 documents, 23 failed attempts, 1.001 s per document. What the number covers: an in-process call, warm, but a document in a legacy format is converted by a LibreOffice or pandoc subprocess that the tool starts itself, so part of the time is that process.
- **pandoc 3.9** - 28 documents, 11 failed attempts, 0.330 s per document. What the number covers: one process per file.
- **LibreOffice 7.4.3.2** - 33 documents, 22 failed attempts, 7.468 s per document. What the number covers: the office suite plus pandoc, one pair of processes per file.

## How this was measured

Each family is scored by its own evaluator over its own corpus. The scales are not the same scale, so no figure of one family is averaged with another's.

- **iWork, archives, data and mail** - anydoc bench coverage manifest, https://github.com/firecrawl/anydoc. 26 mixed samples; samples from the same project that the quality corpus does not contain, including iWork packages, archives and compressed streams.
- **Office formats and e-books** - anydoc bench corpus, https://github.com/firecrawl/anydoc. 63 office files and e-books, of which 40 carry a written ground truth; the sample set of the anydoc project, with a ground truth written for this corpus rather than produced by any engine under test.
- **PDF** - opendataloader-bench, https://github.com/opendataloader-project/opendataloader-bench. 200 PDF documents; derived from DP-Bench and PubLayNet, with a Markdown ground truth for every document.
- **The same documents in fifteen formats** - opendataloader-bench, re-emitted in fifteen formats, https://github.com/opendataloader-project/opendataloader-bench. 200 documents for each format; the ground truth of the PDF suite, written out by pandoc as DOCX, ODT, RTF, HTML, EPUB and PPTX, by the office suite as DOC, PPT and ODP, as a workbook for XLSX, XLS and ODS, and by a generator as the '09 XML generation of Apple Pages, Numbers and Keynote. A workbook carries the structure of the ground truth rather than a column of units: each section of the document is a sheet named after its heading, a table is a real grid of rows and columns, and a paragraph is a row of one cell, so a reader has to recover the table boundaries and the sheet outline that the file states. CSV is not one of the fifteen: it carries no headings and no table boundaries, so the table and heading metrics are zero by construction and ranking structured formats against it would measure the metric rather than the reader.

### Boundaries of the method

**iWork, archives, data and mail**

- A tool that is not installed on the measuring machine is reported as not measured, never as a capability of the tool; an engine missing from this table is one that was not installed.
- Two samples carry no content of their own: `numbers2-file.numbers` holds 0 text cells, 0 numbers and 0 dates, only 69 empty grid cells and 18 checkboxes, and `pages4-file.pages` holds no body text at all - every text run in it is a placeholder, and both the package's own preview and an independent office suite render it as a blank page. An empty answer is the correct answer to them, so they are left out of the denominator instead of being counted for or against any engine - every engine's row is read against the 24 samples that carry content.
- A package that was unpacked is not a document that was read: an engine that opens the archive and prints its members is reported as not supported, not as producing a document. This is why markitdown has no iWork row here - its answer to those packages is the archive's members, up to 2,090,755 characters of them, and the cell text of the document is not among them.
- An archive is read as its members: a ZIP, a tar and a compressed stream are answered with the documents inside them, so an engine that produced an archive is one that returned text from inside it, not one that echoed the container's bytes. LibreOffice and pandoc open no archive at all, and the two compressed-byte answers they give on a `.tar.xz`, a `.tar.bz2` and a `.tar.zst` are text of replacement characters, which this table does not count as a document.

**Office formats and e-books**

- The corpus is small and written for this run: it covers the formats it covers well and says nothing about large or unusual real-world documents. The PDF corpus is the counterpart for those.
- A score appears only where an engine produced a document for the format. A cell that carries no score states why: `n/r` when the engine has no reader for that format, `f` when it declares one and returned no document. The documents a figure covers are printed beside it.
- The table metric reads an HTML `<table>` cell by cell and expands no `colspan`/`rowspan`, while the ground-truth grid is span-expanded: a document whose table merges cells scores below 1 for an engine that emits HTML tables even when the merge is carried faithfully. Two documents of this corpus have merged cells.
- The number of documents a metric covers is printed beside it.
- Time is stated in seconds per document, with what each figure includes: a process started once per file is not a call inside a process that is already warm.

**PDF**

- The `mdapi` answer carries markup the document does not contain: the source's name as a title, one heading per page because that is how `part` addresses a page, and a block of the properties the file declares about itself. That markup is the service's framing rather than a reading of the document, so it is removed before scoring, exactly as the fifteen-format suite removes it, and the metrics then ask whether the document's own text, order, tables and headings came back. The properties are delivered to every caller and stay out of the comparison because they are what the service adds: scoring them would measure that addition rather than the document. No other engine's answer is altered in any way.
- A table is recognised when its columns line up: cells that float without alignment are still emitted as rows of text, so the table score follows what the layout allows rather than the contract. A heading level is inferred from the font size, and a heading set in the body size but bold is placed below every sized heading.
- A heading that wraps onto a second physical line is joined into one heading; a heading split across a page break or a column boundary is not, because the join reads one page's lines.
- A time is shown only for an engine measured here, as a percentage of the shortest time measured: those runs were made on the same machine and the same corpus, in separate runs over one working period, so the ratio compares engines rather than machines and states an order of magnitude, not a controlled benchmark. The same ratio does not carry over to a figure published by another project: those runs used an accelerator where these used a processor, which is why docling reads 30.1 s per page here and 0.762 s per page there. The ratios hold inside one run, never across two.
- Two rows are the bench project's own figures and carry no time: unstructured 0.17.2 in its [hi_res] configuration, and pymupdf4llm 0.2.0. Both are shown because the same engine appears twice with two different results, and leaving one of them out would read as selection rather than as a boundary of version or configuration: pymupdf4llm 0.2.0 scores 0.7316 against 2.1's 0.8664, and unstructured in its default configuration scores 0.6874 against 0.8414 in [hi_res].
- The nutrient row was measured here through the vendor's free batch command-line tool, which converts a corpus in one process: 0.9111 at 0.399 s per page. The project's own publication of the same engine reports 0.885 at 0.008 s per page, which is its SDK called in process rather than a command-line tool. Both figures describe nutrient; they differ by the surface the engine was called through, so the row describes the surface this service would use.
- edgeparse is unpinned upstream: the row names 0.3.0, and a clean install today resolves 0.3.2, which scores 0.8268 on the same corpus - it emits soft hyphens inside words and a tab after numbered headings. The figure belongs to the version named beside it.
- Two marker versions are shown because the newer one does not follow the older one on this corpus: 2.1.0 scores 0.8317 against 1.6.2's 0.8563, and its table score falls furthest (0.6764 against 0.8715). The two rows were also run with different model backends - 2.1.0 through a local llama.cpp build of its model, 1.6.2 through its own default pipeline - so the time ratio between them compares configurations as much as versions.
- The opendataloader hybrid row is 2.5.11 in the bench project's own configuration of it, and the reference published upstream was made with 2.2.1, whose table markup differs: 2.5.11 flattens some tables into a single line of text, so its table score here is far below that reference while its structure score stays high. The figure belongs to the version and configuration named beside it, as with edgeparse.
- Two engines of the other corpora have no PDF reader and are stated as `n/r` on this corpus rather than run: pandoc converts to PDF but not from it, and mammoth reads DOCX alone. anydoc and LibreOffice were added to this corpus for the same reason the other engines are here - they accept a PDF - so the table carries every engine the service is compared with on the one corpus all of them can read.
- anydoc answered no document for three PDFs of the corpus, each of them a page with no text layer that it reported as needing OCR. The evaluator counts a document with no answer as zero, so the row's 0.8665 covers all 200 documents, 197 of which anydoc answered; it is the same accounting every other row here follows.
- LibreOffice emits no Markdown, so its answer is the office suite's own HTML export read as Markdown - the path the service's HTML reader follows - and its figure describes that path. anydoc writes Markdown itself. No other engine's answer passes through another engine.

**The same documents in fifteen formats**

- The `mdapi` answer wraps the document in markup the document does not contain: the source's name as a title, a metadata block, a contents line, one heading per page or slide, and, for an e-book, the reader's own table of contents. That wrapper is removed before scoring, because it is the service's framing rather than the document's text. The heading a workbook carries is not part of it: a sheet's heading is the sheet's own name, which the file states, so it stays in the answer and is scored. No other engine's answer is altered in any way, so every other row carries whatever markup its own pipeline emits.
- Every format in this corpus is written by a converter from one Markdown ground truth, so the heading levels, the lists and the tables in the source are the ones that converter writes and not a typesetter's layout. All the engines read the same files, so the comparison between them holds, while the absolute figures describe this corpus and not a hand-authored office document.
- HTML, RTF, EPUB and PPTX were written with a document title, and a reader emits that title as a heading the ground truth does not carry, so the heading score of every engine is lower on those formats than its reading of the body would suggest. The `mdapi` HTML row is the exception: a heading inside a navigation block or inside a page-level header is returned as plain text when the document has a heading outside those regions, so our row no longer pays for a title that only the generator put there. Every other engine still pays for it, which is why the gap between our row and theirs is wider on this format than the reading of the body alone would explain. EPUB is unaffected: the e-book's navigation document is not in the spine and no chapter of this corpus carries a landmark heading, so the same rule changes none of the 200 files.
- A workbook carries one sheet per section of the document, named after that section's heading, so the heading score of the spreadsheet formats follows the sheet names the file states. A sheet whose grid is one cell wide is delivered as lines of text rather than as a table, because one column states no relationship between cells that the lines do not: the corpus writes a paragraph of the ground truth as a row of one cell, and reading those rows as a table would put a table where the document has prose.
- Excel limits a sheet name to 31 characters and does not allow a colon in it, so a heading longer than that, or one containing a colon, reaches an XLSX or XLS workbook shortened while the same heading reaches an ODS workbook whole. The gap between those rows on the same documents belongs to the container, not to the reader.
- DOC, PPT, ODP, ODS and XLSX are the office suite's own conversion of the DOCX, the PPTX and the ODS, so a legacy reader is compared on a file that suite wrote, not on a file a legacy tool produced. A document with a real legacy layout can read differently.
- The three Apple formats are the '09 XML generation of Pages, Numbers and Keynote, written by a generator from the same ground truth as the rest of the corpus. The IWA generation of the same applications is a different container and is not in this corpus, so a modern Apple document is not described by these three rows.
- TEDS is measured on the documents of the corpus that carry a table, so its mean covers those documents and not all 200, and a format whose answer contains no table scores zero on it.
- The engines compared are docling 2.145.0, unstructured 0.27.36, markitdown 0.1.8, pandoc 3.9, LibreOffice 7.4.3.2, anydoc and mammoth 1.12.0. Each was run on every format it has a reader for, and a format it cannot read is stated as `n/r` rather than left blank: mammoth reads DOCX alone, pandoc takes DOCX, ODT, RTF, HTML and EPUB, markitdown reads DOCX, PPTX, XLS, XLSX, RTF, HTML and EPUB, unstructured reads DOCX, DOC, PPTX, PPT, XLS, XLSX, ODT, RTF, HTML and EPUB, anydoc reads every format here except HTML and the three Apple packages, LibreOffice reads every format here except EPUB, and docling reads all fifteen. All seven were installed and run here, so no figure in this table is quoted from another project.
- LibreOffice emits no Markdown of its own: its answer is the office suite's HTML export for text and spreadsheets and its PDF export for presentations, read as Markdown by the path the service uses for that format. The empty `<span id>` anchors that export writes before a heading are removed before that read: they carry no text, and a reader of the page drops them. Every other engine writes Markdown itself.
- One figure covers fewer than the 200 files of its format: on NUMBERS LibreOffice answered 196. The evaluator counts a file with no answer as zero, so that figure describes the format as a whole rather than only the files the engine answered.

## Beyond the score tables

A score answers "how close is the converted text to the source". These are the answers a caller needs and no score covers; each is stated as what this service answers and what the compared engines offer.

- **A locked payload** - the password is named in the body field `password` or the `X-MDAPI-Content-Password` header, and the payload opens - PDF, OOXML and ODF office files, ZIP, 7z, encrypted iWork. Two codes tell the two failures apart: `password_required` when no password came, `wrong_password` when it came and did not open the payload; both answer 415 and neither spends a paid call. The secret is never read from the query string, never cached and never logged. Against the engines compared here: opendataloader (`--password`) and docling (`--pdf-password`) take a password for a PDF; markitdown, pandoc and the headless LibreOffice conversion have no way to supply one for an office file, and none of them opens a locked archive.
- **One part of a large document** - `part=N` returns one page, slide, sheet or chapter of the converted result, and `result=meta` lists the parts without converting the document again. Against the engines compared here: opendataloader (`--pages`) and docling (`--page-range`) convert a page range of a PDF; neither addresses parts of an office file or of an archive, and each re-converts the file to do it.
- **A search over a converted document** - `find` answers the addresses of the matches - the part and the offset inside it, with the page box where the source carries layout - and never the matched text, so a large document is not sent back to be searched. Against the engines compared here: none of them offers a search over a document. opendataloader and docling emit bounding boxes in their JSON output, but as a by-product of converting the whole file, not as an answer to a query.
- **One subtree of the source** - `select` names an HTML or XML element, or a path into JSON, and the answer is that subtree alone. Against the engines compared here: none of them offers it.

The methods and the quality of recognition are revised continuously, and this page is re-measured with the same evaluator after each revision.

## Links

- **About service:** https://mdapi.io/about
- **Privacy:** https://mdapi.io/privacy
- **Contact:** https://mdapi.io/contact
- **Agent entry point:** https://mdapi.io/AGENTS.md or https://mdapi.io/agents.md or https://mdapi.io/agent.md
- **Agent access:** https://mdapi.io/auth.md
- **API documentation and conversion:** https://mdapi.io
- **MCP server manifest:** https://mdapi.io/mcp
- **Health check:** https://mdapi.io/health
- **Documentation index:** https://mdapi.io/llms.txt
- **Full API documentation:** https://mdapi.io/llms-full.txt
- **AI discovery:** https://mdapi.io/.well-known/ai-discovery.json or https://mdapi.io/ai-discovery.json
- **AI Agent discovery:** https://mdapi.io/.well-known/agent.json or https://mdapi.io/agent.json
- **A2A Agent card:** https://mdapi.io/.well-known/agent-card.json or https://mdapi.io/agent-card.json
- **ACP manifest:** https://mdapi.io/.well-known/acp.json or https://mdapi.io/acp.json
- **x402 payment manifest:** https://mdapi.io/.well-known/x402.json or https://mdapi.io/x402.json or https://mdapi.io/.well-known/x402
- **OpenAPI specification (JSON):** https://mdapi.io/.well-known/openapi.json or https://mdapi.io/openapi.json
- **OpenAPI specification (YAML):** https://mdapi.io/.well-known/openapi.yaml or https://mdapi.io/openapi.yaml
- **MAPI specification:** https://mdapi.io/.well-known/mapi.md or https://mdapi.io/mapi.md
- **Skill specification:** https://mdapi.io/.well-known/skill.md or https://mdapi.io/skill.md
- **ARD manifest:** https://mdapi.io/.well-known/ard.json or https://mdapi.io/.well-known/ai-catalog.json or https://mdapi.io/ard.json
- **Agent Plugins manifest:** https://mdapi.io/.well-known/plugin/plugin.json or https://mdapi.io/.well-known/plugin.json
- **Agent Plugins MCP config:** https://mdapi.io/.well-known/plugin/mcp.json
- **Agent Plugins conversion skill:** https://mdapi.io/.well-known/plugin/skills/mdapi-conversion/SKILL.md
- **API documentation pages:** https://mdapi.io/docs
- **Example pages:** https://mdapi.io/docs/examples

## External Links

- **github.com** https://github.com/mdapiio/mdapi.io
- **skills.sh** https://www.skills.sh/mdapiio/mdapi.io
- **skillsmp.com** https://skillsmp.com/creators/mdapiio/mdapi.io
- **clawhub.ai** https://clawhub.ai/mdapiio
- **x.com** https://x.com/mdapiio

## Disclaimer

**The service is provided "AS IS".**


> mdapi.io is an edge-native service-transport primitive for AI, autonomous agents, and the Web4 ecosystem.
