Engineering guides / AI and realtime voice integration
By Arte Soft engineering · 24 October 2026
Voice latency diagnostics: find where conversational delay begins
“The assistant is slow” describes an experience, not a cause. A useful latency audit separates the end of the user’s speech from the first audible response and identifies which stage can actually be improved.
Draw the timeline before changing settings
Mark capture start, speech end, turn submission, response arrival and audible playback. Use a monotonic clock within each process, and be careful when comparing timestamps across devices and servers. A user hears the total delay, but an engineering decision needs its components. Otherwise a lower silence threshold can look like a faster model while simply cutting off the end of sentences.
Distinguish continuous delay from occasional stalls
Collect multiple turns across cold starts, warm sessions and network changes. Report distributions and individual bad traces rather than a single average. Note whether the assistant delayed every answer or only the first one after connecting. Repeated long tails can point to setup, audio route activation, buffer scheduling or retries, and each has a different remedy from changing generation parameters.
Inspect playback as closely as the network
Receiving audio promptly does not prove it was heard promptly. A queue may contain stale samples, a device graph may be restarting, or a route may use a different sample rate. Record queued duration and the transition into audible playback. When a response is interrupted, verify that discarded audio does not remain ahead of the next answer. A clean transport trace can coexist with a frustrating listening experience.
Use representative environments
Compare speaker and headphones, Wi-Fi and mobile data, and a lower-powered supported phone alongside a recent one. Keep background work visible in the test notes. If the application uploads an image before a spoken response, include that dependency in the trace instead of blaming voice alone. State the distance and acoustic conditions when human judgments are part of the assessment.
Turn findings into bounded changes
The audit deliverable should include an annotated timeline, ranked causes and experiments that isolate them. A proposed change needs a predicted benefit and a regression risk: shorter endpointing may interrupt users; smaller buffers may expose underruns. Re-run the same utterances after each change. Do not combine several optimizations and then attribute an improvement to whichever one sounded most plausible.
Capture one slow turn properly
- Keep the utterance and connection conditions stable while collecting capture, submission, first-response and speaker-start markers.
- Annotate whether the perceived silence began before submission or after data reached the device; these imply different fixes.
- After an optimization, repeat a difficult long-pause sample to verify that apparent speed did not come from prematurely ending the user’s turn.
Define acceptance without promising impossible silence
Agree acceptable response ranges for the intended task and explicitly handle long waits in the interface. A visible connecting or thinking state is more useful than pretending all delays have disappeared. GPT Realtime Assistant is a relevant published Arte Soft reference, while each client audit needs its own traces. The final report should distinguish measured application improvements from provider variability and unsupported conclusions about every future network condition.
Relevant Arte Soft work
GPT Realtime Assistant — our own published product is our own product or engineering project. It provides context for the work we discuss here; it is not a claim of identical results for every client deployment.
Sources and further reading
Need help with Voice latency audit?
Send the current app, system or brief, the behaviour you need and your constraints. We can agree a discovery, implementation or repair scope with concrete acceptance criteria.
Discuss this with Arte Soft