# Human Latency Conversational Turns for Spoken Avatar Systems

arXiv: https://arxiv.org/abs/2404.16053 · HTML: https://mlsanity.com/paper/2404.16053/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 24 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 3 of 3)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A BERT Baseline for the Natural Questions](https://mlsanity.com/paper/1901.08634/) (2019)

## Built on by (most cited first, 5 of 5)

- [Building a Taiwanese Mandarin Spoken Language Model: A First Attempt](https://mlsanity.com/paper/2411.07111/) (2024)
- [Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architectures](https://mlsanity.com/paper/2504.11750/) (2025)
- [Dialogue Response Prefetching Based on Semantic Similarity and Prediction Confidence of Language Model](https://mlsanity.com/paper/2508.04403/) (2025)
- [Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents](https://mlsanity.com/paper/2511.07397/) (2025)
- [A Social Robot with Inner Speech for Dietary Guidance](https://mlsanity.com/paper/2505.08664/) (2025)

## Abstract (arXiv)

A problem with many current Large Language Model (LLM) driven spoken dialogues is the response time. Some efforts such as Groq address this issue by lightning fast processing of the LLM, but we know from the cognitive psychology literature that in human-to-human dialogue often responses occur prior to the speaker completing their utterance. No amount of delay for LLM processing is acceptable if we wish to maintain human dialogue latencies. In this paper, we discuss methods for understanding an utterance in close to real time and generating a response so that the system can comply with human-level conversational turn delays. This means that the information content of the final part of the speaker's utterance is lost to the LLM. Using the Google NaturalQuestions (NQ) database, our results show GPT-4 can effectively fill in missing context from a dropped word at the end of a question over 60% of the time. We also provide some examples of utterances and the impacts of this information loss on the quality of LLM response in the context of an avatar that is currently under development. These results indicate that a simple classifier could be used to determine whether a question is semantically complete, or requires a filler phrase to allow a response to be generated within human dialogue time constraints.