Engineering guides / AI and realtime voice integration
By Arte Soft engineering · 24 October 2026
Voice interruption and barge-in: stop audio and preserve the right context
An assistant has not really stopped when its animation changes but queued speech keeps playing. Barge-in requires coordinated control over capture, response generation, playback and the conversation context used for the next answer.
Specify what counts as an interruption
Decide whether any detected speech can interrupt, whether a button is required and how short noises are treated. Distinguish a deliberate new question from the assistant’s output leaking back into the microphone. On a speaker route, barge-in and echo control must be evaluated together. Otherwise an aggressive interruption policy can repeatedly cancel the assistant because it hears its own voice.
Follow cancellation through the full pipeline
Stopping remote generation does not automatically remove samples already buffered in the client. Clearing a local player does not necessarily stop a response still arriving from the service. List the actions for each transport and represent their completion in a state machine. An interrupt action should be repeatable without leaving the session unable to accept its next turn.
Align memory with what the person heard
A conversation can become confusing when the model remembers an entire answer that the user stopped halfway through. Establish whether the transport handles unplayed output automatically or whether the client must truncate it. Store the right relationship between visible transcript and audible content. This is particularly important when the next question is “what did you just say?” or refers to a step that never reached the speaker.
Handle races instead of hiding them
Test interruption before the first audio chunk, during playback and immediately after output ends. Add rapid repeated presses and a simultaneous typed message. Give each response a stable identity so late events cannot append audio or text to a newer turn. A stale completion event should not change the interface from listening back to speaking after the user has already started another request.
Make acceptance audible and inspectable
Use synthetic output with recognizable numbered phrases, interrupt at a known point and check that later phrases are not heard. Review event order, queued duration and final conversation state. Then repeat on real speaker and headset routes. An automated assertion that a cancel method was called is useful, but it does not establish that the user’s device stopped making sound.
Interruption evidence to request
- Keep a response identifier and the playback position at the moment of interruption, with a trace showing queued output was discarded.
- Check what the next turn remembers about the interrupted response, especially instructions that were generated but never heard.
- Add a repeated interruption test so a duplicate cancel event cannot strand the application between listening and speaking.
Scope a repair around evidence
For an existing voice app, a focused engagement can map the cancellation pathways and fix one reproducible race before redesigning everything. The handover should include test sequences and ownership of playback buffers. Arte Soft’s GPT Realtime Assistant provides published product context for these discussions. No implementation should promise flawless double-talk on every device without documenting the routes, versions and acoustic environments used in acceptance.
Relevant Arte Soft work
GPT Realtime Assistant — our own published product is our own product or engineering project. It provides context for the work we discuss here; it is not a claim of identical results for every client deployment.
Sources and further reading
Need help with Conversation interruption implementation?
Send the current app, system or brief, the behaviour you need and your constraints. We can agree a discovery, implementation or repair scope with concrete acceptance criteria.
Discuss this with Arte Soft