The pitch for PDF-to-Markdown is that Markdown is what your model wants: headings, lists, tables, math. That is true. The question is whether the converter actually produces those structures, or just text with some newlines.
What breaks, in order of damage
1. Tables flattened into text
A financial table rendered as running text loses the relationship between a number and its column header. A retriever then returns a paragraph containing “12,450” with no way to know whether that is revenue, a balance, or a footnote. For RAG this is worse than a missing chunk: it is a confident wrong answer.
2. Multi-column reading order
Two-column academic papers are the standard test. A parser that reads in text-stream order interleaves the columns, producing sentences that belong to different paragraphs. Reading order has to be reconstructed from layout, not from the order characters appear in the file.
3. Formulas
Equations become images, or a sequence of garbled symbols. A model that cannot see the formula will answer confidently about a paper it did not really read.
4. Page-break tables
A table continuing onto the next page arrives as two fragments, often with the second fragment missing its header. Chunked separately, neither fragment is retrievable in a useful way.
What to look for in the output
- Tables as Markdown tables, with one row per row — not as sentences.
- Headings preserved as
#levels, so chunking can follow document structure. - Math as LaTeX, inline and display.
- Page anchors, so a retrieved chunk can be traced back to the source page.
A five-minute acceptance test
- Take ten documents that resemble your real corpus, not the cleanest ones.
- Count how many tables survived as tables.
- Check one table cell: does the number match the source exactly?
- Check one formula: is it LaTeX, an image, or garbage?
- Check chunk boundaries: do headings line up with sections?
If two or more of those fail, the pipeline will not improve with a better embedding model. The problem is upstream.
Where SheetParser fits
It produces structured Markdown with headings, Markdown tables and LaTeX math, and keeps a page reference for extracted content. For tables that do not survive — page breaks, borderless layouts — the disputed cells are listed rather than silently flattened, so you can exclude a bad chunk instead of indexing it.
Try it on the paper or report that your current parser mangles: the first file each day is free up to 10 pages, and paid plans run files up to 600 pages.