Voice agents break in the runtime, and no better LLM will fix that

Two hundred milliseconds at one end, a full second at the other, and the post says there is no universal answer anywhere between them ^[9]. That is a factor of five ^[1] on a parameter with no correct value, only two ways of being wrong: too short and the agent talks over someone who was mid-sentence, too long and a two-line exchange starts to drag ^[10]. Nobody ships a fix for that by changing model vendors.

The timer cannot be tuned into correctness because it is measuring the wrong thing. Voice activity detection asks a frame-level question about the audio: is there a human voice in here, or a fan, a keyboard, traffic, a door slamming ^[6]. It does not know what was said, so it cannot separate a finished thought from a pause inside one ^[7]. The post's example is a request that stops after "I'd like to book an appointment" and then continues "for tomorrow afternoon", at which point the pause retroactively meant nothing ^[14]. The acoustics never contained that information. Hence the hybrid the post points to, where production systems weigh acoustic signal and semantic content together, with LiveKit's turn detection documentation offered as the example ^[11].

Interruption handling escalates the same way. The volume-threshold version of the problem is binary. The real one has at least four answers: a genuine interruption, a backchannel, background noise, or the agent hearing its own voice come back ^[13], which is twice the state space VAD was built to report ^[2]. Get the backchannels wrong and the agent stops mid-sentence because a listener said "uh-huh", and the conversation reads as broken ^[12]. That decision has to resolve while the agent is still speaking, which is also when it has the least room to spend.

Then the part that is not a classifier at all. Barge-in requires the agent to stop itself mid-sentence, hold the context that still matters, and pick up from whatever the user just said ^[4], and the post is explicit that doing it well takes tight coordination across several components at once ^[15]. The runtime also owns the handoff to a human when one is needed ^[16]. A model swap touches one box in that diagram, and it is not the box where any of this lives.

Worth being clear about what the source is. Both of its examples of production practice are LiveKit documentation ^[11][13], so read them as evidence that someone builds this way rather than as a survey of the field. And the central diagnosis, that the hardest voice problems today sit in the runtime rather than the language model ^[3], is an argument, not a measurement. It is a well-formed one. The request-response shape of a text chatbot, message in and response out ^[2], is exactly the shape that makes model shopping feel like the main lever, and a voice agent does not have that shape: the audio arrives as a stream and the system has to detect, transcribe, reason, call tools, synthesise and play back while still listening ^[1].