mlsanityData to 7 Oct 2026

Controlling Backchannels in Streamable Full-duplex Models

arXiv 2609.29418 ↗First posted 24 Sep 2026cs.CL cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Backchannels, brief acknowledgements like "uh-huh" produced while the other party may still be talking, are central to natural conversation, but full-duplex spoken dialogue models rarely model them explicitly. We introduce a lightweight backchannel head that predicts, from a full-duplex model's own hidden states, when a backchannel should begin. Once this probability crosses a tunable threshold, a backchannel is force-decoded. Attached to both a 7B (PersonaPlex) and a 1B (F-Actor) model, it generalizes across scale. Probing confirms the hidden states anticipate real human timing, and generation evaluation shows more frequent, better-timed backchannels. Human raters judge the resulting backchannels on par with real ones.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 25 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

21 of 25 references.

References · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  4. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  5. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 94
  6. 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 57
  7. 2025Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and ASTcited by 51
  8. 2025SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Modelcited by 32
  9. 2025NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inferencecited by 14
  10. 2017Yeah, Right, Uh-Huh: A Deep Learning Backchannel Predictorcited by 12
  11. 2026Pearmut: Human Evaluation of Translation Made Trivialcited by 11
  12. 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Modelscited by 9
9 more
  1. 2024Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projectioncited by 8
  2. 2025Sidon: Fast and Robust Open-Source Multilingual Speech Restoration for Large-scale Dataset Cleansingcited by 8
  3. 2026TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialoguecited by 4
  4. 2026VoXtream2: Full-stream TTS with dynamic speaking rate controlcited by 3
  5. 2026DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretrainingcited by 2
  6. 2026Endpoint Anticipation for Low-Latency Spoken Dialoguecited by 2
  7. 2025Multilingual and Continuous Backchannel Prediction: A Cross-lingual Studycited by 1
  8. 2026Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuningcited by 1
  9. 2026Synchronization and Turn-Taking in Full-Duplex Speech Dialogue Modelscited by 1

Built on by

Not cited yet.