# Does Listening Matter? Backchanneling and Nodding in AI Clone

arXiv: https://arxiv.org/abs/2608.19527 · HTML: https://mlsanity.com/paper/2608.19527/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Responsive Listening Head Generation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2112.13548/) (2021)
- [PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models](https://mlsanity.com/paper/2602.06053/) (2026)
- [Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems](https://mlsanity.com/paper/2205.15060/) (2022)
- [Multilingual Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2403.06487/) (2024)
- [Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection](https://mlsanity.com/paper/2410.15929/) (2024)
- [Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational Agents](https://mlsanity.com/paper/2407.01824/) (2024)
- [Hear You in Silence: Designing for Active Listening in Human Interaction with Conversational Agents Using Context-Aware Pacing](https://mlsanity.com/paper/2602.06134/) (2026)
- [AI Twin: Enhancing ESL Speaking Practice through AI Self-Clones of a Better Me](https://mlsanity.com/paper/2601.11103/) (2026)
- [Migratable AI: Effect of identity and information migration on users perception of conversational AI agents](https://mlsanity.com/paper/2007.05801/) (2020)
- [Real-time Generation of Various Types of Nodding for Avatar Attentive Listening System](https://mlsanity.com/paper/2507.23298/) (2025)
- [Cloning the Self for Mental Well-Being: A Framework for Designing Safe and Therapeutic Self-Clone Chatbots](https://mlsanity.com/paper/2601.15465/) (2026)

## Abstract (arXiv)

AI clones that imitate a specific person typically reproduce what the person says and how they sound, but not how they listen. We investigate whether adding multimodal listening behaviors gives such a clone more presence and authenticity. We integrated verbal backchannels and head nodding, driven by real-time prediction models, into an AI clone equipped with voice cloning and LLM-based responses. In a within-subjects study (N=35), adding these behaviors significantly improved the perceived attentiveness of the avatar, the sense of talking with the real person, and the feeling of co-presence. These results indicate that AI clone fidelity should extend beyond voice and response content to include interactive listening behavior.