Engineering guides / AI and realtime voice integration
By Arte Soft engineering · 24 October 2026
Acoustic echo cancellation: diagnose an assistant that hears itself
When an assistant recognizes its own spoken answer as a new request, prompt changes are rarely enough. Investigate how speaker output reaches the microphone, which audio processing path is active and whether playback bypasses that path.
Identify acoustic feedback before treating language symptoms
Repeated translations, invented follow-up questions and duplicated phrases can be downstream symptoms of feedback. Reproduce the issue with a known synthetic sentence played through the application speaker while nobody speaks nearby. Compare captured input and playback timing. If the recognized words follow the assistant’s own output, investigate the signal path before adjusting model instructions or blaming a language pair.
Give the audio graph one owner
Capture and playback should have a clear lifecycle and route policy. Inventory audio engines, players, WebRTC components and speech synthesizers that can activate the system session. Two independently configured subsystems may each appear correct in isolation yet conflict when combined. Record when voice processing is enabled and whether graph reconstruction preserves that setting after a route change or an interruption.
Choose processing with the interaction in mind
Apple exposes voice-processing controls, but enabling one API is not proof that the complete application is protected. Verify the active route and the actual playback path. Muting capture during playback can be a temporary half-duplex fallback, though it prevents natural interruption. A full-duplex requirement must be tested as such rather than satisfied by hiding feedback with a microphone gate.
Make the test reproducible
Use a fixed utterance, device placement, volume range and quiet-room procedure. Add double-talk cases where a real user interrupts during output. Capture only the diagnostic data needed and avoid retaining personal conversations by default. Compare residual feedback, intelligibility and turn behavior; a quieter recording is not automatically better if the user’s voice was suppressed along with the echo.
Require evidence across routes
Speakerphone, receiver, wired audio and Bluetooth do not necessarily share the same processing behavior. Repeat checks after headphones disconnect, the app returns to foreground and the session restarts. The acceptance record should show which routes passed, which use a fallback and which remain excluded. Avoid a universal decibel claim based on one room, one volume level and one device.
Echo regression essentials
- Use a fixed synthetic phrase and record whether any of its words become an unintended new user turn.
- Interrupt deliberately during the phrase to verify that a successful echo reduction did not simply disable the microphone for the entire answer.
- Repeat after route changes and record volume and placement, so a later engineer can reproduce the acoustic conditions rather than guessing.
Frame the engagement as diagnosis plus a verifiable fix
Bring the source project, a reproducible sequence and the affected device models. Arte Soft can discuss this work against its published GPT Realtime Assistant product experience, without claiming an independently certified audio system. Useful deliverables include a graph ownership map, event instrumentation, a regression script and the limits of the final interaction. The goal is a dependable conversation, not simply a successful microphone permission prompt.
Relevant Arte Soft work
GPT Realtime Assistant — our own published product is our own product or engineering project. It provides context for the work we discuss here; it is not a claim of identical results for every client deployment.
Sources and further reading
Need help with Echo and audio graph review?
Send the current app, system or brief, the behaviour you need and your constraints. We can agree a discovery, implementation or repair scope with concrete acceptance criteria.
Discuss this with Arte Soft