Skip to content

Build1 publisherNot yet confirmed elsewhere3 min readPublished

The re-encode that quadrupled a GIF was passing every test that looked at pixels

A pipeline that turned 1074 KB into 4881 KB was not losing to the compressor. It was decoding away the interframe differencing the source already had.

The Engineer · Build desk

How we use AISend a correction

Illustration accompanying The re-encode that quadrupled a GIF was passing every test that looked at pixels
Generated illustration

What happened

  • Re-encoding a 288-frame pendulum GIF took it from 1074 KB to 4881 KB, and a 70-frame turret went from 116 KB to 1232 KB.
  • The inflation also happened with no crop applied, pointing at the encoder rather than the compressor.
  • A changed-pixel threshold for choosing delta versus keyframe was written and then removed after the measurements contradicted it.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint With a median of 90%, this cannot be sold as a compression gain; the benefit sits almost entirely in the files whose structure was being destroyed, and everything else moves a rounding error.
  • capability Encoding two candidates and keeping the smaller replaces a rule nobody can state correctly, which is available to any pipeline willing to spend the CPU on measurement instead of prediction.
  • exposure Typed-array APIs that reach for .buffer will read past a view and return output that looks right, so this class of defect clears code review and visual QA and only shows up in byte counts.
  • contradiction The repaired sizes and the summary band cannot both describe the same runs, so the headline result is only as good as an accounting the writeup has not published.

The check that would have caught this is not one most pipelines run. Every frame the old encoder emitted was individually correct and the animation played correctly [9], so a visual diff passes it, and so does a hash of the decoded frame buffers. What went missing was structure, not pixels. A GIF is a sequence of image blocks, each with its own disposal method and local palette, and nothing requires a block to cover the whole canvas or to be a complete picture [6]. The pendulum source used that: a static rig and a dark background left alone, bytes spent on a bob occupying a few percent of the pixels [8]. The re-encode wrote full-canvas opaque keyframes instead and paid for that background 287 extra times [10].

That is the general shape. Any step that decodes a format to a canonical intermediate and re-serialises it discards the source's encoding decisions, and if those decisions were the compression, no output check based on appearance can see the loss. The size ratio is the detector, which is how this one surfaced at all: 1074 KB to 4881 KB on a 288-frame file [2], 116 KB to 1232 KB on a 70-frame one [3], with the same inflation on a pure re-encode with no crop [5].

The most useful part of the writeup is the heuristic that was built and then deleted [13]. The intuitive rule is to skip the delta when too much of the frame changed, and the measurements inverted it: the file that changed twice as much was the one that should have used deltas, the file that changed less was the one that should not [14]. LZW pays for runs, not for pixels, so what decides the outcome is how scattered the movement is, and no threshold over a changed-pixel count can see scatter [15]. So the encoder measures instead. Three sample frames per file, encoded both ways, winner kept [16], with both candidates appended after an identical frame 0 so the header, global palette and loop block cancel out of the comparison [17].

Across ten Wikimedia Commons animations [18], output runs 15% to 100% of the previous encoder's, median 90%, nothing bigger [1]. That median is the number worth holding: at least half the set gained 10% or less [21]. This is not a compression improvement with a long tail. It is a repair with a very short one, and the two repaired cases move 6.7x and 3.8x [22][23].

The numbers do not fully close. The fixed turret is 184 KB and the fixed pendulum 1292 KB [19], which are 59% and 20% above the input sizes quoted earlier in the same piece [25][26], while the summary says all ten finished at 17% to 79% of their sources [24]. The writeup does not say which files that band covers.

The palette footgun is the same failure mode in miniature: hand `quantize()` a subarray view and it reads the whole underlying buffer, builds the palette from the wrong pixels, and returns something plausible with no error [20].

What to watch

  • Whether a per-file table reconciles the 17% to 79% band against the 184 KB and 1292 KB figures, which sit above the inputs quoted for the same two files.
  • Whether a three-frame probe misreads animations whose motion scatter changes late in the sequence, such as a molt or a libration cycle.
  • Whether gifenc gains sub-rectangle image descriptors, which would remove the full-canvas workaround and the CPU it costs.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories