Prompting will not fix pronunciation: taking the audio back out of the video model

A developer running a Turkish character-video pipeline has stopped trying to prompt a generative video service into correct pronunciation and has moved speech synthesis out of the model entirely, according to Part 3 of their writeup on dev.to ^[1][19][20]. The useful part is not the workaround but the measurement that killed the prompting approach: a word-level transcription confidence gate that no amount of instruction could satisfy ^[7][10][11]. The visual side was never the complaint. Character identity held, lighting was good, camera movement looked natural ^[2]. What came out of the mouth was the problem: repeated phrases such as "bilim insanlari ile birlikte de bilim insanlari" ^[3], the model quietly rewriting a sentence containing "insulin" into cautious science language ^[4], invented readings of loanwords ("eureka" spoken as "urika") ^[5], and a key word, "yayla", compressed into 0.26 seconds until it sounded like "aile" ^[6]. The gate is faster-whisper large-v3, run with Turkish forced and word timestamps on: transcribe the clip, and if any word's probability falls under 0.80, reject the clip ^[7][8]. It worked, and it throttled production ^[9]. The prompt fixes tried were the obvious ones: do not reword, no stutter, no repeated words, speak slowly and over-articulate every syllable, and stretch the clip from 10 to 12 seconds. Each lowered the rejection rate, none ended it, because the model being asked for clean speech was the same model rendering the pixels ^[10][11]. The first instinct was audio-driven video: Wan2.2-S2V, HunyuanVideo-Avatar, EchoMimicV3, OmniAvatar, all of which take your audio and generate lips to match ^[12]. With no GPU on the server, EchoMimicV3 went onto a notebook service with free GPU hours, and failed three times ^[13]. First ffmpeg could not open the dataset wav, so trimming moved into Python with librosa and soundfile at 16 kHz ^[14]. Then paths did not mount where expected, so the notebook was made to walk /kaggle/input and find the first .wav and .jpg itself ^[15]. Then the kernel died on return code -9 ^[16]. The log had the actual diagnosis: 220 missing keys, all in the audio injection layer, meaning the base video weights loaded and the audio conditioning weights did not, so audio would not have driven the image even if the process had survived ^[17]. That path was closed and filed for a future GPU environment ^[18]. The replacement splits the two jobs the video model was doing: scene and identity, where it is strong, and speech, where it is not ^[19]. Text goes to edge-tts in Turkish, through the 0.80 Whisper gate, then a base clip is lip-synced with Wav2Lip, restored with GFPGAN, tagged and queued ^[20]. The commercial engine still narrates site articles for archive tone consistency; spending monthly character quota on a 12-second video line was judged wasteful, and edge-tts is free with two Turkish voices ^[21]. One finding transfers beyond this pipeline. At a rate of -4%, the gate flagged three words from their own TTS (kart 0.73, Para 0.74, Falin 0.77); at -8% everything passed ^[22]. Their conclusion is that rushed speech is unintelligible regardless of who generated it ^[23]. Character variety now comes from pitch shifting the same voice ^[24]. The cost has moved downstream. Wav2Lip does a 12-second vertical clip in about three minutes on CPU ^[25], but it renders the mouth at 96x96, which turns to mush on an 800-pixel face for a pipeline already accused of looking artificial ^[26]. GFPGAN repair runs about 2.7 seconds per frame on CPU ^[27], which means restoration overtakes the lip-sync pass at roughly 67 frames ^[28]. Because the gate now runs on the TTS output before any clip work, a rejection costs seconds instead of a generation pass ^[29].