Nuance Labs raises $50M to replace the avatar's four-model pipeline with one

Fangchang Ma, Nuance Labs' co-founder and chief executive, told Business Insider that the face that stops moving three words into a user's sentence comes from how the products are put together, not from a fault in any one part ^[5][8]. "If you use existing AI avatars, when they're listening, they're not reacting, or they're just doing random things," Ma said ^[9].

A typical avatar runs four stages. Speech to text, then a language model, then text to speech, then a face animated to match the audio ^[6]. Ma said replies come back late because the system processes everything step by step ^[7]. Four stages means three handoffs, and Nuance's answer deletes all three: one model, audiovisual in and audiovisual out, perceiving the user's stream while streaming a response back ^[10][19]. "We decided to build this on one system, on one model," Ma said. "There's audio/video in and audio/video out." ^[12]

What exists today is a demo the company describes as a work in progress ^[13]. The account includes no latency measurement and no benchmark against the assembled pipeline ^[20]. So the case for one model over four rests on Ma's explanation and on that demo. Nnamdi Iregbulem of Lightspeed, which also backed the seed round ^[2], said of the founders that "they've turned that into a single model that follows you in real time" ^[17].

Ma is aiming at sales and customer service, coaching, professional training and education, including practising an interview or learning a language over a video call ^[15]. Those split on one question: where the value arrives. A customer asking when a refund clears wants the answer, and a beat of dead air before it costs them almost nothing. Someone rehearsing a job interview is buying the listening half, the nod at the right moment and the flicker when an answer rambles. Ma said the model can demonstrate understanding in the moment, while the user is still speaking ^[11].

That question decides whether waiting is worth it. Where the value lands in the reply, the assembled pipeline can carry the product, and shipping it now is the honest call. Where the value lands during the user's turn, the pipeline you wire together cannot deliver it, and the vendor making the claim has a preview coming and no product ^[14]. Either way, the useful figure to have before that preview arrives is task completion and repeat use on what you run now, because that preview can only be measured against your own baseline.