A GPU SQL Engine Lost to One CPU Thread Because a Dispatcher Constant Was 128x Too Small

A maintainer of a Metal-backed GPU SQL engine ran his own benchmark on ten million rows and lost to a single CPU thread walking a hash map by 1.7x ^[2][3][4]. The fix was one integer, but the transferable part is the method: he swept the workload's only free parameter before opening anything else, and the shape of the result named the bug class ^[6].

The engine is a DuckDB community extension that runs aggregates and GROUP BY on Apple Silicon through Metal, according to a writeup published on dev.to ^[1][24]. At 10M rows and 1,024 groups, single-threaded std::unordered_map posted a 14.65 ms median wall time against 25.39 ms for the Metal path on an M4 Max ^[2][3].

Varying group cardinality across six points showed CPU time rising gently, from 14.62 ms at 8 groups to 30.37 ms at 65,536 ^[7]. The GPU line did not behave like that: 6.50 ms at 256 groups, 25.09 ms at 1,024, then partial recovery to 13.30 ms at 65,536 ^[8]. His reading is that a curve means the algorithm degrades, while a cliff followed by recovery means a branch chose differently ^[9]. The engine had been printing that choice to stderr all along: radix-opt at 256 groups, slot-lock at 1,024 ^[10]. Note what the grid alone bought before any code was read. Two adjacent sample points localised the fault to a 4x-wide interval ^[2].

The dispatcher selected the slot-lock kernel whenever the expected group count sat between kSlotLockMinGroups = 1,024 and kSlotLockSafeCap = 16,000,000 ^[11]. That kernel builds 4,096 fixed hash partitions with threadgroup-resident slot tables, a layout intended for millions of groups; hand it 1,024 groups and the partitions collapse onto a handful of slots while thousands of threads serialise behind slot locks ^[12].

An existing environment override let both kernels be raced at fixed cardinality rather than trusting the comment above the constant ^[13]. Radix-opt stayed between 7.32 and 8.17 ms from 256 through 65,536 groups, a spread of about 1.12x, while slot-lock fell from 27.67 ms to 13.33 ms over the same range ^[14][15][1]. They crossed at 131,072 groups, 11.24 ms against 10.32 ms ^[14][15][16]. The threshold was therefore 128x below the measured crossover, and everything in the 64x-wide band between them got the slower kernel, by up to 3.1x ^[17].

Setting the constant to 131,072 moved auto-dispatch from 24.59 ms to 8.13 ms at 1,024 groups and from 23.62 ms to 7.85 ms at 2,048, gains of 3.02x and 3.01x, with 2.59x at 8,192 and 1.63x at 65,536 ^[18][19]. Against the single CPU thread, the 1,024-group case went from 0.60x to 1.81x, and 65,536 groups from 2.32x to 3.77x ^[20]. He reports 96 of 96 unit checks and 72 of 72 SQL tests still passing, with the benchmark checking every result against a CPU reference on each run ^[21]. The measurement table was left in the comment above the constant ^[23].

The scale check is the honest part. Crossover sits near 131K groups at 1M rows and drifts to roughly 196K at 50M, so a fixed 131,072 is conservative at the high end, costing about 8% in one cell while being far better across the 1K to 64K band ^[22].

Two things to watch. First, any dispatcher threshold in your own stack that was reasoned rather than measured, because those constants live precisely where nobody benchmarks. Second, whether that upward drift with row count eventually forces this threshold to become a function of input size instead of a literal ^[22]. All numbers here are the author's own, from one machine, unreplicated ^[24].