Build1 publisher3 min readPublished
A benchmark harness with a ground-truth word list caught an OCR pipeline shipping empty PDFs
For months the pipeline answered in four or five seconds with the right page count and no error, while recall against a known word list sat at zero. Five faults were stacked downstream of recognition, three of them silent by design.
The Engineer · Build desk
What happened
- For months the site's OCR handed back a PDF with the right page count after a believable four or five seconds and reported no error, and the text layer inside that PDF was empty.
- Nobody complained, because a searchable PDF with no searchable text looks like a searchable PDF until you press Ctrl+F, and it took a benchmark harness with a ground-truth word list to notice.
- Recognition itself was perfect, and all five faults sat downstream of it, each one hiding the next until it was peeled back.
- Three-hundred-dpi scans failed on page one while hundred-and-fifty-dpi scans went all the way through, an inversion of the usual assumption that large files are the hard case.
- Six of the twelve languages offered in the dropdown could not produce a text layer at all, and Polish, the one language the author needed, was not in the list.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Monitoring on error rates cannot see this class of fault, because "no recognisable words" is a legitimate answer for a blank page; the pipeline has to be judged on the content of its output, not its exit status.
- decision Test fixtures now have to be chosen against the code paths they enter, since anything under about 175 dpi never reaches the resize path at all and passes for reasons that have nothing to do with the code under test.
- cost A library default that is off without saying so moves the debugging cost onto whoever reads its constants file, and here that cost was paid in scan quality and resolution checks first.
- precedent Round-tripping the artefact, writing the words and reading the PDF back, becomes the admission test each language has to pass, and it is what keeps Arabic off the list while its accuracy is unmeasured.
`if (result.words.length > 0)` wrapped the code that drew the invisible text layer, and in tesseract.js v7 `result.words` does not exist; words live at `data.blocks[].paragraphs[].lines[].words[]` [12]. Coerced through `(result.words || [])` the undefined field became an empty array, so the guard never fired, no text layer was drawn, and nothing threw [13]. A blank page produces that same result legitimately, and the code reported both the same way [14].
The next level down had the same shape. Reading from `data.blocks` instead left recall at 0%, because `data.blocks` was present in the result object with the value `null` [15]. Blocks are off by default in `tesseract.js/src/worker-script/constants/defaultOutput.js`, and you turn them on with a third argument: `await worker.recognize(blob, {}, { blocks: true })` [16]. An absent field sends you to read the types; a null one answers your question, with no stack trace and nothing to grep for [17]. Two hours went into the scan quality, the render resolution and the language data before anybody looked at the output flags, and the scan had been fine the whole time [18].
The crash upstream of that had a threshold. pdf.js reaches its `DOMCanvasFactory` default only through `ImageResizer`, which engages once a page exceeds `MIN_IMAGE_DIM`, 2048 pixels, and that default calls `document.createElement('canvas')` inside a Web Worker, where `document` does not exist [6][7]. A4 at 300 dpi is 2481 x 3507 pixels; at 150 dpi it is 1240 x 1754 [8]. The long side at 150 dpi sits 294 pixels under the threshold [1], and it grows about 11.7 pixels per dpi, so the resize path engages above roughly 175 dpi and a lower-resolution fixture never touches it [2]. The threshold also depends on the machine, and the author wrote that here "works on my machine" was not a figure of speech: "It was literally true and completely useless." [9] The fix is a canvas factory built on `OffscreenCanvas`, which a worker does have, injected where pdf.js expects its own [10]. With the crash gone, 300 dpi scans completed and still had no text [11].
The last fault reached users. `page.drawText()` called without an explicit `font` makes pdf-lib fall back to a standard font with WinAnsi encoding [19]. Out of ąćęłńóśźż exactly one character survived, ó, and every word in Russian, Japanese, Chinese, Arabic and Hindi threw an encoding exception into an empty `catch`, so the pages came out clean and wordless [20]. The interface advertised "multi-language recognition, including languages with diacritics" [21].
What the harness measures matters as much as the fact that it runs. Recognition was already perfect [4], so a check on tesseract's own confidence would have passed every time; recall against a known word list is what moved [3][15]. The font work got the same treatment: a real embedded font registered through fontkit and subset into the output [22], then a round trip to verify it, writing the words, reading the PDF back and comparing, which cut the list to the eleven languages the font provably encodes [23]. Chinese, Japanese and Hindi came back as NUL bytes and were removed, and Arabic survives the round trip but its accuracy is unmeasured, so it stays out [24].
What to watch
- Whether Arabic gets an accuracy measurement and rejoins the list of eleven encodable languages.
- Whether pdf.js ships a worker-safe default canvas factory so ImageResizer stops needing an injected one.
- Whether the harness gains a per-stage assertion that fails loudly when blocks comes back null.