# Yeah, Right, Uh-Huh: A Deep Learning Backchannel Predictor

arXiv: https://arxiv.org/abs/1706.01340 · HTML: https://mlsanity.com/paper/1706.01340/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2017.

## Built on by (most cited first, 12 of 12)

- [Applying General Turn-taking Models to Conversational Human-Robot Interaction](https://mlsanity.com/paper/2501.08946/) (2025)
- [When can I Speak? Predicting initiation points for spoken dialogue agents](https://mlsanity.com/paper/2208.03812/) (2022)
- [Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection](https://mlsanity.com/paper/2410.15929/) (2024)
- [Exploring Semi-Supervised Learning for Predicting Listener Backchannels](https://mlsanity.com/paper/2101.01899/) (2021)
- [TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening](https://mlsanity.com/paper/2202.08216/) (2022)
- [F-Actor: Controllable Conversational Behaviour in Full-Duplex Models](https://mlsanity.com/paper/2601.11329/) (2026)
- [Joint Learning of Context and Feedback Embeddings in Spoken Dialogue](https://mlsanity.com/paper/2406.07291/) (2024)
- [Spoken DialogSum: An Emotion-Rich Conversational Dataset for Spoken Dialogue Summarization](https://mlsanity.com/paper/2512.14687/) (2025)
- [Multilingual and Continuous Backchannel Prediction: A Cross-lingual Study](https://mlsanity.com/paper/2512.14085/) (2025)
- [Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning](https://mlsanity.com/paper/2604.16622/) (2026)
- [CMIS-Net: A Cascaded Multi-Scale Individual Standardization Network for Backchannel Agreement Estimation](https://mlsanity.com/paper/2510.17855/) (2025)
- [Controlling Backchannels in Streamable Full-duplex Models](https://mlsanity.com/paper/2609.29418/) (2026)

## Abstract (arXiv)

Using supporting backchannel (BC) cues can make human-computer interaction more social. BCs provide a feedback from the listener to the speaker indicating to the speaker that he is still listened to. BCs can be expressed in different ways, depending on the modality of the interaction, for example as gestures or acoustic cues. In this work, we only considered acoustic cues. We are proposing an approach towards detecting BC opportunities based on acoustic input features like power and pitch. While other works in the field rely on the use of a hand-written rule set or specialized features, we made use of artificial neural networks. They are capable of deriving higher order features from input features themselves. In our setup, we first used a fully connected feed-forward network to establish an updated baseline in comparison to our previously proposed setup. We also extended this setup by the use of Long Short-Term Memory (LSTM) networks which have shown to outperform feed-forward based setups on various tasks. Our best system achieved an F1-Score of 0.37 using power and pitch features. Adding linguistic information using word2vec, the score increased to 0.39.