ClearPix cuts an in-browser video job from 95 seconds to 9 by replacing ffmpeg.wasm with WebCodecs

Per frame, the shipped loop is short. MediaBunny opens the upload as a BlobSource, takes the primary video track and checks canDecode before touching a frame ^[13]. VideoSampleSink then yields decoded frames as an async iterator ^[13]. Each sample is drawn to a canvas, closed, and passed to an encode step that snapshots the canvas to JPEG, with at most eight snapshots in flight ^[14]. The draw call also applies rotation and flip metadata, so phone footage comes out upright with no extra code ^[15].

I think the sample.close() call and the cap of eight explain most of the flat memory profile. A decoded frame is released as soon as it is drawn. The queue of pending snapshots has the same bound for a short clip as for one at the new 120-second limit ^[6]. The authors credit the streaming single-pass design ^[6]. One overhead survived the rewrite. The post lists JPEG round-trips among the costs that replacing ffmpeg.wasm removed ^[11], yet the loop still encodes every frame to JPEG ^[14]. What went away is the copy of each frame into and out of ffmpeg.wasm's virtual filesystem ^[4].

The gain is a factor of about 10.6, or 86 seconds per run ^[1][2]. It was measured on one job, watermark removal, on the team's Macs, where VideoToolbox handles H.264 encoding in dedicated hardware ^[5]. Part of the old baseline was a cold start. The post says the 33MB fetch costs "the first several seconds of the session" on a cold cache ^[3], while WebCodecs initialization downloads nothing and reports ready at once ^[9]. A returning user with the wasm build already cached would have seen a smaller gap. Workload shape matters too. The authors' rule is that "pipeline overhead and model overhead are the same order of magnitude, and pipeline overhead is usually cheaper to fix" ^[12]. A tool whose run time is mostly inference would get less from the same migration.

The backend switch is careful work. A ?vbackend=ffmpeg|webcodecs URL parameter or a localStorage key overrides the dispatcher ^[16]. Forcing webcodecs skips the probe and hard-routes, so a hardware failure throws during testing instead of dropping silently to the fallback ^[17]. I would ship the same default. Automated coverage is thinner. Playwright's bundled Chromium has no H.264 codec, so the end-to-end suite forces the ffmpeg path ^[10].

Freezing the fallback is the right call in their context. Streaming it "would cost a week of work on a path almost nobody runs," the post says ^[18]. That judgement rests on "almost nobody." The post does not give the share of sessions that fail the hardware probe.