Cactus Compute fits a 16.9MB speech model into the CPU runtime it built for tool calling

Whistle ships in Needle's .cact model container and runs on Needle's CPU inference engine ^[7]. A team already deploying one of Cactus's tool-calling models gets speech input without taking on a second runtime or model format ^[7]. According to Cactus, the engine builds for 17 platform targets, including RISC-V, MIPS, WebAssembly and WASI ^[16].

The model is an encoder-decoder ^[5]. A log-mel front end and a convolutional stem feed an eight-block audio encoder, and a decoder with gated cross-attention writes the text ^[5]. The company says the decoder can be loaded at different depths while the encoder always runs at full depth ^[6]. Trimming the decoder makes each token cheaper, but every clip still pays for the whole encoder pass ^[6]. The launch material does not say which decoder depth produced the published figures, or how long a ten-second clip takes from first sample to finished transcript.

On Cactus's figures, Whistle reaches its first token about 6.6 times sooner than Whisper base and decodes about five times faster ^[19][20]. The speed test used ten seconds of audio and reported time to first token separately from decoding ^[15]. I think reporting the two apart is correct for a model of this shape. The first number includes the encoder pass, and the second is decoder work alone.

Needle 2, the release Whistle builds on, was about running tool-calling models on low-cost devices ^[8]. For the speed ratios to carry over to cheaper hardware, both models would have to slow down by the same factor on a weaker CPU. The published timings cover one Apple M4 Pro, against 17 build targets ^[14][16].

Accuracy splits three sets each way among the head-to-head comparisons Cactus names ^[24]. Whistle takes both LibriSpeech sets and FLEURS, while the company's own benchmark page gives TED-LIUM, AMI and the MLS average to Whisper base ^[11]. The LibriSpeech margins are 0.59 and 0.51 points of word error rate ^[21][22]. A half-point lead transfers to a product only if its audio resembles LibriSpeech more than the sets Whisper base wins. On FLEURS the gap is wider, at 3.1 points ^[23].

The test design shows care. According to Cactus, its word-error runs span 86,174 utterances, and the company compared the test sets with the data Whistle was trained and validated on ^[13]. That contamination check is the first thing I'd ask any vendor about. Comparison models ran on their official runtimes with default settings, and published Whisper and Moonshine results were used where available ^[12]. Runtimewire's report notes that the figures remain company-reported ^[18].

For a voice-command feature on hardware a team already targets with Needle, I'd bench Whistle before Whisper base. Adding it means one more model in a container format the app already loads ^[7]. For recordings that look like TED-LIUM, AMI or MLS, Cactus's own table favours Whisper base ^[11].