pdfium's text page drops the 0 Tz line that pypdf still returns

pdfium and pypdf are answering different questions about the same bytes. pdfium's text extraction goes through a text page built from layout ^[6]. pypdf walks the `Tj` operators in the content stream and returns what it finds ^[7]. Put `0 Tz` on a line and every glyph gets zero width, so nothing appears on the page ^[9]. That line never reaches pdfium's text page, and `count_chars()` does not count it ^[5]. pypdf returns it on a second line after the visible body text ^[8]. The obvious guard, a check for a suspiciously small font, does not fire here, because the glyph height is untouched ^[9].

At `1 Tz` pdfium does return the text, and the character box is then 0.08 pt wide for a 12 pt letter drawn as a hairline ^[10]. Working back from that width, the unscaled advance was about 8 pt ^[21]. The extraction boundary sits at exactly zero width: the 0.08 pt version is unreadable to a person and present in both extractors ^[10].

Each library departs somewhere from what a reader sees ^[24]. On `0 Tz`, pdfium matches the page and pypdf adds a line nobody can see ^[4]. On `/ActualText` it goes the other way, with pdfium returning the replacement string and pypdf returning the glyphs that were drawn ^[11]. All of it is within the PDF specification, and the dev.to post's author says neither library is wrong ^[3].

"So don't trust a single extractor in front of an LLM. Run two, normalize, and treat the difference as the signal," the author wrote ^[14]. That covers the two cases above. The two where the outputs agree slip past it: a rewritten `/ToUnicode` table makes an on-screen A extract as B in both readers, and mappings to invisible Unicode tag characters pass through both untouched ^[12]; text inside a Form XObject that is never invoked with `Do` is read by neither, though pypdf's `extract_xform_text` gets it ^[13]. A diff between the two default APIs flags two of the four documented mismatches ^[20].

The cases were run on PDFs the author wrote as raw syntax, in a builder of about 60 lines that stacks objects and writes the xref table by hand ^[17], with one font and one visible line per page and one construct added per file ^[18]. "If you generate test PDFs with a library, you lose control over how the text is hidden," the author wrote ^[16]. The post does not report how often these constructs turn up in real documents ^[22]. For any of this to reach your pipeline your PDFs have to carry those constructs, and the published outputs are pinned to Python 3.12 with pypdfium2 5.12.1, pdfium build 7947 and pypdf 6.15.0 ^[15].