Firecrawl opened pdf-inspector: PDF in Markdown in milliseconds, without OCR

Firecrawl, a startup from Y Combinator that makes tools for extracting data from websites, has opened the sources of pdf-inspector: a Rust library that in milliseconds determines whether the text in front of you is PDF or a scan, and turns text documents into pure Markdown without OCR at all. According to the team's estimates, about 54% of PDF in real problems are text-based, and there is simply no point in running them through the slow and expensive OCR.

If you have ever fed a neural network PDF - a contract, a report, a book - you know this feeling: the file seems to open, but the text from it is not copied or is copied in porridge. Classic parsers either choke on tables or run each page through OCR, wasting seconds and money. pdf-inspector hits exactly this pain: first, it quickly looks at what kind of file it is, and connects OCR only if it is a real scan.

First classification, then extraction

In one pass, the library assigns PDF to one of four types - TextBased, Scanned, ImageBased or Mixed - and gives a confidence from 0 to 1 plus a page route: which to read as text, which to send to OCR. Classification takes 10–50 milliseconds. Next, for text pages, there is extraction with coordinates, fonts and reading order, and the output is Markdown with headings, lists, tables and links.

Numbers that are impressive

On the opendataloader-bench out of 200 PDF pdf-inspector disassembled the entire case in 0.47 seconds. For comparison: pymupdf4llm - 17 seconds, markitdown - 16 seconds. At the same time, it is the highest in quality: the overall score is 0.875 versus 0.735 for pymupdf4llm and 0.589 for markitdown. It recognizes tables much better - 0.814 versus 0.401 and 0.273.

What else can he do?

Multi-column newspaper layouts, right-to-left text, CID fonts, broken encoding detection. Separately - selective OCR: if individual scanned pages are stuck in PDF, you can run only them through OCR, locally, through PP-OCRv6 Small, and not the entire document.

Where's the catch?

Honestly: for real scans and photographs of documents, OCR is still needed, and here the pdf-inspector is not magic - it just doesn’t spend OCR on something that doesn’t need it. Complex layouts, where formulas and layout are mixed with text, can also work. And this is a library, not a ready-made service: to use it, you have to write a little code.

There are bindings for Python, Node.js and browser-based WASM - the same Rust parser works directly in the browser without a server. The repository on GitHub has already gained almost 17 thousand stars. If you often carry PDF in a neural network, it’s worth a look.