A GPU delegate that returns nothing: why exception-based fallbacks miss the real failure

A team building an offline virtual mirror for hair salons found that MediaPipe's ImageSegmenter with the GPU delegate was created without throwing, ran in 5 ms, and returned zero hair pixels on a Samsung A56 (Exynos 1580, Mali GPU) - both the confidence mask and the category mask came back empty, every time ^[7]. Their fallback to CPU triggered only on exceptions, there were no exceptions, and so the app simply looked slow instead of broken ^[8].

The constraints are worth stating because they are common. No frame leaves the device: model, fonts and runtime are all local, since a CDN request would break offline use and leak the salon's IP to a third party ^[2]. And salons do not buy flagship tablets, so the target is whatever mid-range Android is on the counter ^[3]. On a Pixel 8 Pro the same build ran at 9 fps, which the author calls poor but working; on the A56 there was no colour at all ^[5][6]. All figures come from diagnostics built into the engine and measured on real devices, according to the write-up ^[4].

The instructive part is the benchmark trap. The GPU path measured 96 times faster than CPU precisely because it was not doing any work, so timing alone would have selected the broken path ^[9]. Taken with the 5 ms GPU figure, that implies a working CPU pass on the order of 480 ms ^[27], consistent with the 437 ms of blocked main thread the team measured for the synchronous segmentForVideo call on Exynos ^[19]. The stated lesson is to check the output, not the exit code; the team now measures mask coverage and surfaces it in diagnostics ^[10].

The fallback rule is more careful than "retry on empty". After five consecutive empty masks, having never seen a good one in the whole session, the engine rebuilds on CPU ^[11]. The distinguishing condition is "never seen a mask", not "empty right now", because an empty mask is entirely normal when nobody is in front of the lens ^[12]. That is the difference between a broken delegate and an empty chair, and it has to be encoded explicitly. On the A56 the rebuild took the app from 2 fps to 14 fps with colour applied ^[13], a factor of seven ^[26].

Observability is not free either. An outputCategoryMask left enabled from an earlier experiment became a significant part of the frame budget ^[14].

Once the GPU path was honest, the profile of the Pixel 8 Pro's 9 fps was unflattering to the surrounding code rather than the model: 6 ms of inference against 126 ms of the team's own work ^[15], roughly 4.5 percent of the frame ^[25]. That work was 66 ms in a JavaScript loop recolouring pixels one at a time ^[16] and two GPU-to-CPU readbacks per frame, one of them costing 123 ms ^[17]. The author reports this is the shape of most "the ML model is too slow" complaints they have received ^[28]. The loop went into a fragment shader, free because the pixels were already on the GPU ^[18]; the readback went away by passing MediaPipe the same canvas and taking the mask with getAsWebGLTexture() ^[21]; and segmentation moved to one frame in every adaptive step, drawing the video on all of them ^[20].

Two constraints to watch if you copy this. MediaPipe recycles that texture underneath you, which produced colour flicker until the team copied it into their own texture with a shader ^[22]. And MediaPipe will not start inside a type: "module" worker, dying with ModuleFactory not set because its WASM loader registers the factory via importScripts() ^[24] - so the reflex of moving a blocking call off-thread is not available unless you use a classic worker. The team expected to need a worker and did not; decoupling cadence was enough ^[20].