Skip to content

Build Not yet confirmed elsewhere1 publisher 3 min read Published

A hardcoded /Length turns a byte-exact PDF into a parse error

Real headers and an exact byte count pull against each other, and every format loses the fight differently. One generator author walked 25 of them and catalogued the bugs that browsers forgive and CI parsers refuse.

The Engineer · Build desk

How we use AI Send a correction

Illustration accompanying A hardcoded /Length turns a byte-exact PDF into a parse error
Generated illustration

What happened

  • The obvious approach, dd from /dev/urandom, hits a 25 MB target exactly and produces a file that any format-sniffing validator rejects instantly, because it is not a PDF.
  • In the generator's first PDF, the content stream object declared a hardcoded /Length of 44 while the stream it wrapped actually measured 38 bytes.
  • For DOCX, filler appended after the closing tag of word/document.xml opened in Word but made python-docx raise an lxml error about extra content at the end of the document.

Why it matters

  • contradiction Verifying with the lenient consumer is exactly what let both PDF defects through, so the discovery moves out of the generator's own tests and into whoever runs the strict parser downstream.
  • constraint Exact sizing is only available where a format designates a region readers must ignore, which makes byte-precision a per-format privilege rather than a feature you can promise across a catalogue.
  • decision The choice of acceptance parser now carries the correctness argument: pypdf, python-docx, openpyxl and PIL in the loop are the difference between a green build and a fixture that fails at the customer.
  • precedent For formats whose parts cross-reference each other, cloning a validated artifact and patching a few entries becomes the defensible default, and building from spec becomes the choice that needs justifying.

A strict PDF reader does not go looking for `endstream`. It seeks past the `stream` keyword, reads exactly as many bytes as `/Length` declares, and expects the closing keyword at the cursor. Declare 44 over a stream that measures 38 [5] and it consumes six bytes that were never part of the stream [22], then errors where the keyword should have been [7]. Lenient viewers scan instead of trusting the field, which is why the same bytes rendered in a browser and were refused by Acrobat and pypdf [6]. The second defect in the same object was quieter: `/Resources << >>` was empty while the content stream drew text with `/F1`, and viewers substitute a font silently rather than complain [8].

Both fields are the same category of mistake. `/Length` and the cross-reference offsets restate information the assembled bytes already contain, so the fix is to measure rather than predict, recording `body.length` for each object as it is appended [9]. That is a one-line change in the assembly loop, and it is the part of this writeup that generalises furthest.

Where padding may go is not the generator's choice. It is whatever region the format designates as ignorable. For PDF that is after `%%EOF`, which a conforming reader is required to skip [10]. For DOCX it is an XML comment before the closing tag, which python-docx and openpyxl both accept while the byte count stays exact [12]. For PNG it is a `tEXt` ancillary chunk before `IEND` with a correct CRC-32, because decoders skip ancillary chunks they do not know [13]. For JPEG it is `FFFE` COM markers holding up to 65,533 bytes each, chained, placed immediately after SOI [14]. That capacity sets the shape of the work: 25 MB of COM padding is at least 401 chained markers [23], and every library that walks the marker chain walks all of them. The author reports that padding near EOI decoded unreliably across libraries, that moving it fixed the problem, and that he never root-caused why [25]. That is an unexplained workaround, and saying so plainly matters more than inventing a tidy explanation.

The floors table describes one generator's output shape, with numbers specific to it: 69 bytes for JSON, 114 for XML, 170 for SVG, 600 for PDF [26]. Those numbers transfer only when your minimum document matches the same top-level structure, the same key or element names, and the same declarations; the failure mode, not the byte count, is what actually carries over to other generators. A 20-byte JSON request sits 49 bytes under that floor [27], and the generator quietly emitted whitespace of exactly the right size that failed `JSON.parse()` [16]. Rounding up and saying so is the honest answer.

PPTX is where generation from spec stopped paying. A minimal deck needs presentation.xml, presProps.xml, a slide master, a slide layout and a theme, with the master and layout referencing each other, and a small error produces a repair prompt [17]. The author now ships a known-good deck as a static asset, clones its blank slide, and patches three files [18]. Sizing comes from a binary search on `ppt/pad.xml`, an orphan part covered by the default XML content type and referenced by nothing [19]. The byte budget now rests on a reported behaviour: PowerPoint ignores unreferenced parts entirely [19].

The through-line is the acceptance test. Every bug listed passed the test the author was running [21]. A fixture that only opens in Word is a fixture for a human. The pipeline is run by something else [24].

What to watch

  • Whether a strict OPC validator, or any non-PowerPoint consumer, rejects the orphan ppt/pad.xml part the PPTX byte budget depends on.
  • A root cause for JPEG COM padding decoding unreliably near EOI, which would say whether post-SOI placement is a fix or a coincidence.
  • Whether callers get a machine-readable signal when a requested size is rounded up to the format floor, rather than a log line.

Clarity's read

What the record supports and how the coverage leans. The claims behind it follow.

Reality

Evidence62
Adoption14
Hype gap−8
Incentives35
Confidence58
Why these scores

Claim ledger

Ranked by verification strength, evidence, and original report placement.

  1. [1]

    The obvious approach, dd if=/dev/urandom of=test.pdf bs=1M count=25, yields exactly 25 MB but produces a file that any format-sniffing validator rejects instantly because it is not a PDF.

    Reported SupportedView cited source
  2. [2]

    The author built a generator for 25 file formats that emits real headers at exact byte sizes.

    Reported SupportedView cited source
  3. [3]

    Real headers and an exact byte count are requirements that fight each other, and every format loses the fight differently.

    Reported SupportedView cited source

Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.

Add to ChatGPTAdd to Claude

Sources

1 independent publisher whose own reporting we read for this story.

  1. dev.to

    1 article · August 29, 2026

    I generated 25 file formats at exact byte sizes. Here's every way it broke.

Share your take

Let Clarity write the post for you.

Signed-in readers get a short post drafted on this story in the register they choose — narrative, analytical, or a direct position — editable to the last word before it goes anywhere. The share buttons at the top of this story work without an account.

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Entities

Loading related stories