A hardcoded /Length turns a byte-exact PDF into a parse error
Real headers and an exact byte count pull against each other, and every format loses the fight differently. One generator author walked 25 of them and catalogued the bugs that browsers forgive and CI parsers refuse.
The Engineer · Build desk

What happened
- The obvious approach, dd from /dev/urandom, hits a 25 MB target exactly and produces a file that any format-sniffing validator rejects instantly, because it is not a PDF.
- In the generator's first PDF, the content stream object declared a hardcoded /Length of 44 while the stream it wrapped actually measured 38 bytes.
- For DOCX, filler appended after the closing tag of word/document.xml opened in Word but made python-docx raise an lxml error about extra content at the end of the document.
Why it matters
- contradiction Verifying with the lenient consumer is exactly what let both PDF defects through, so the discovery moves out of the generator's own tests and into whoever runs the strict parser downstream.
- constraint Exact sizing is only available where a format designates a region readers must ignore, which makes byte-precision a per-format privilege rather than a feature you can promise across a catalogue.
- decision The choice of acceptance parser now carries the correctness argument: pypdf, python-docx, openpyxl and PIL in the loop are the difference between a green build and a fixture that fails at the customer.
- precedent For formats whose parts cross-reference each other, cloning a validated artifact and patching a few entries becomes the defensible default, and building from spec becomes the choice that needs justifying.
A strict PDF reader does not go looking for `endstream`. It seeks past the `stream` keyword, reads exactly as many bytes as `/Length` declares, and expects the closing keyword at the cursor. Declare 44 over a stream that measures 38 [5] and it consumes six bytes that were never part of the stream [22], then errors where the keyword should have been [7]. Lenient viewers scan instead of trusting the field, which is why the same bytes rendered in a browser and were refused by Acrobat and pypdf [6]. The second defect in the same object was quieter: `/Resources << >>` was empty while the content stream drew text with `/F1`, and viewers substitute a font silently rather than complain [8].
Both fields are the same category of mistake. `/Length` and the cross-reference offsets restate information the assembled bytes already contain, so the fix is to measure rather than predict, recording `body.length` for each object as it is appended [9]. That is a one-line change in the assembly loop, and it is the part of this writeup that generalises furthest.
Where padding may go is not the generator's choice. It is whatever region the format designates as ignorable. For PDF that is after `%%EOF`, which a conforming reader is required to skip [10]. For DOCX it is an XML comment before the closing tag, which python-docx and openpyxl both accept while the byte count stays exact [12]. For PNG it is a `tEXt` ancillary chunk before `IEND` with a correct CRC-32, because decoders skip ancillary chunks they do not know [13]. For JPEG it is `FFFE` COM markers holding up to 65,533 bytes each, chained, placed immediately after SOI [14]. That capacity sets the shape of the work: 25 MB of COM padding is at least 401 chained markers [23], and every library that walks the marker chain walks all of them. The author reports that padding near EOI decoded unreliably across libraries, that moving it fixed the problem, and that he never root-caused why [25]. That is an unexplained workaround, and saying so plainly matters more than inventing a tidy explanation.
The floors table describes one generator's output shape, with numbers specific to it: 69 bytes for JSON, 114 for XML, 170 for SVG, 600 for PDF [26]. Those numbers transfer only when your minimum document matches the same top-level structure, the same key or element names, and the same declarations; the failure mode, not the byte count, is what actually carries over to other generators. A 20-byte JSON request sits 49 bytes under that floor [27], and the generator quietly emitted whitespace of exactly the right size that failed `JSON.parse()` [16]. Rounding up and saying so is the honest answer.
PPTX is where generation from spec stopped paying. A minimal deck needs presentation.xml, presProps.xml, a slide master, a slide layout and a theme, with the master and layout referencing each other, and a small error produces a repair prompt [17]. The author now ships a known-good deck as a static asset, clones its blank slide, and patches three files [18]. Sizing comes from a binary search on `ppt/pad.xml`, an orphan part covered by the default XML content type and referenced by nothing [19]. The byte budget now rests on a reported behaviour: PowerPoint ignores unreferenced parts entirely [19].
The through-line is the acceptance test. Every bug listed passed the test the author was running [21]. A fixture that only opens in Word is a fixture for a human. The pipeline is run by something else [24].
What to watch
- Whether a strict OPC validator, or any non-PowerPoint consumer, rejects the orphan ppt/pad.xml part the PPTX byte budget depends on.
- A root cause for JPEG COM padding decoding unreliably near EOI, which would say whether post-SOI placement is a fix or a coincidence.
- Whether callers get a machine-readable signal when a requested size is rounded up to the format floor, rather than a log line.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence62
- Adoption14
- Hype gap−8
- Incentives35
- Confidence58
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The obvious approach, dd if=/dev/urandom of=test.pdf bs=1M count=25, yields exactly 25 MB but produces a file that any format-sniffing validator rejects instantly because it is not a PDF.
- [2]
The author built a generator for 25 file formats that emits real headers at exact byte sizes.
- [3]
Real headers and an exact byte count are requirements that fight each other, and every format loses the fight differently.
- [4]
Stated use cases include a 25 MB PDF to test an upload limit, a 105-page DOCX for pagination, and a 1 GB binary to benchmark throughput.
- [5]
The generator's first PDF emitted a content stream object with a hardcoded /Length 44, while the actual stream is 38 bytes.
- [6]
Browser viewers rendered the malformed PDF fine; Acrobat rejected it, and so did pypdf.
- [7]
A strict reader trusts /Length, reads 44 bytes, sails past endstream, and hits a parse error.
- [8]
A second bug in the same object: /Resources << >> was empty while the content stream referenced /F1; a page that draws text with an undeclared font is invalid even though lenient viewers substitute one silently.
- [9]
The fix is to compute stream lengths and xref offsets from the assembled bytes rather than predicting them, accumulating each object's offset as body.length before appending it.
- [10]
PDF padding goes after %%EOF, the one region a conforming reader is required to ignore, which is what makes an exact byte count possible without corrupting the document.
- [11]
An Office file is a ZIP containing word/document.xml; padding appended after the closing tag opened in Word but made python-docx raise lxml.etree.XMLSyntaxError: Extra content at the end of the document.
- [12]
Putting the padding inside an XML comment before the closing tag makes python-docx and openpyxl both accept the file while the byte count stays exact.
- [13]
PNG padding goes in a tEXt ancillary chunk inserted before IEND with a correct CRC-32; decoders skip unknown ancillary chunks, so the image still decodes.
- [14]
JPEG uses FFFE COM markers, each holding up to 65,533 bytes, chained for larger targets, placed immediately after SOI rather than near EOI.
- [15]
Verification used strict readers: pypdf's PdfReader raises on malformed structure and the author asserts /F1's BaseFont is /Helvetica, while PIL's Image.verify() catches CRC and chunk errors.
- [16]
Asked for a 20-byte JSON file, the generator quietly fell back to emitting whitespace, producing a file of exactly the right size that failed JSON.parse().
- [17]
PPTX needs presentation.xml, presProps.xml, a slide master, a slide layout and a theme, with the master and layout referencing each other; getting it slightly wrong makes PowerPoint show a repair prompt.
- [18]
The author ships a known-good minimal deck as a static asset, fetches it at runtime, clones its blank slide N times, and patches [Content_Types].xml, presentation.xml.rels and presentation.xml.
- [19]
Exact PPTX sizing comes from a binary search on an orphan part, ppt/pad.xml, covered by the default XML content type and referenced by nothing, because PowerPoint ignores unreferenced parts entirely.
- [20]
The author's conclusion for PPTX: borrowing a validated artifact beat generating one from spec.
- [21]
The author's summary is that every bug described passed the test he was running.
- [22]
A declared /Length of 44 over a 38-byte stream makes a trusting reader consume 6 bytes beyond the end of the stream data.
- [23]
Filling a 25 MB target with JPEG COM padding requires at least 401 chained markers.
- [24]
The author argues the entire audience for generated Office files is automated pipelines that parse them, so a fixture that only opens in Word is useless to CI.
- [25]
Padding placed near the end of a JPEG decoded unreliably across libraries; the author never fully root-caused why, and moving it made the problem disappear.
- [26]
Empirically measured floors for the author's output shape: JSON 69 bytes, XML 114 bytes, SVG 170 bytes, PDF 600 bytes; below those, round up and tell the user.
- [27]
A 20-byte JSON request is 49 bytes below the author's measured 69-byte JSON floor.
- [28]
The PDF floor is about 8.7 times the JSON floor in the same generator.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toI generated 25 file formats at exact byte sizes. Here's every way it broke.
1 article · August 29, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.