Switching runtimes cut a three-second Bengali clip from 80 seconds to two on the same phone

Eighty seconds for three seconds of audio is about 27 times the length of the utterance ^[1], and the encoder pass accounted for 60 to 80 of those seconds on six CPU threads ^[10]. Parameter tuning came first: `single_segment = true`, `n_max_text_ctx = 64`, and `audio_ctx = 512` so the encoder ran over about 16 seconds of context instead of 30 ^[11]. The decoder sped up and the encoder settled at 55 to 60 seconds ^[12]. Taking the midpoints, roughly 47 percent less audio bought roughly 18 percent less encoder time ^[2]. Reaching two seconds from there needed another 29-fold cut ^[3]. An attempt to wire in the OpenCL GPU backend was a dead end ^[13].

The 40-fold gain came from the runtime. sherpa-onnx comes from the k2/Kaldi team, ships as an Android AAR and runs ONNX Runtime internally ^[14], and with the same turbo model files it transcribed in about two seconds ^[15], less time than the clip itself ran ^[5]. Eighty divided by two is 40, so the stated factor is those two measurements and nothing else ^[4]. For the number to transfer you need a Pixel 10 with a Tensor G5 on Android 16 ^[2], utterances of about the same length, and the same export of the same weights. The whisper.cpp side ran q5_0 GGML quantisation on six threads ^[8][10]; the write-up does not state the thread count for the sherpa-onnx run.

Three settings had to be right before any of that worked, and two of them fail without complaint. `featureDim` is 128 because turbo uses 128 mel bins where v1 and v2 used 80 ^[16]. `tailPaddings` has to be -1 for auto-detect, because 0 silently produces empty output ^[17]. Language, in v1.13.7, accepted exactly one value: `"bn"` returned silence with no error or warning, which the author traced to GitHub issues about ISO code handling in that release ^[19], `"bengali"` and `"<|bn|>"` each crashed with `Invalid language` ^[20], and the empty string was the only thing that ran ^[21]. A runtime that answers a bad setting with an empty string costs more debugging than one that crashes, and this integration offered both.

Auto-detect then puts the project back where the small model left it. Whisper steers its decoder with special language prefix tokens ^[18], and with no token set the internal classifier picks Hindi at random on short Bengali utterances ^[21]. The small model erred in the same direction: it heard Bengali phonemes and wrote them in Devanagari, so "মাকে ফোন করো" came out as "माकी फोन करो", phonetically close and useless for matching Bengali contact aliases ^[4].

Model size did not predict Bengali quality either. Medium produced fragmented output with random punctuation, stray English words and inconsistent script ^[5]; an explicit prompt, "Transcribe the following Bengali speech:", improved it noticeably, and it still weighed 769 MB ^[7]. "I am not sure why medium performs worse than small for Bengali short-form audio, but it does," the post's author wrote ^[6].

Zipformer gets the credit for the build that finally worked, after frame-size crashes and Android scoped-storage problems, according to the post's own summary ^[22]. Its detailed figures are for the Whisper attempts. Two things went wrong with Whisper on this phone, and changing the runtime fixed the latency while leaving the language classifier picking Hindi on short utterances ^[15][21].