# Oh, Jeez! or Uh-huh? A Listener-aware Backchannel Predictor on ASR Transcriptions

arXiv: https://arxiv.org/abs/2304.04478 · HTML: https://mlsanity.com/paper/2304.04478/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 28 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 2 of 2)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Lexico-acoustic Neural-based Models for Dialog Act Classification](https://mlsanity.com/paper/1803.00831/) (2018)

## Built on by (most cited first, 6 of 6)

- [Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion](https://mlsanity.com/paper/2401.14717/) (2024)
- [Joint Learning of Context and Feedback Embeddings in Spoken Dialogue](https://mlsanity.com/paper/2406.07291/) (2024)
- [Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning](https://mlsanity.com/paper/2604.16622/) (2026)
- [Open Your Model's Eyes: Video and Context-Aware Multimodal Backchannel Prediction](https://mlsanity.com/paper/2607.22729/) (2026)
- [Conversational Rubert for Detecting Competitive Interruptions in ASR-Transcribed Dialogues](https://mlsanity.com/paper/2407.14940/) (2024)
- [Multi-Party Backchannel Prediction: a Diagnosis, a Benchmark, and a Ceiling](https://mlsanity.com/paper/2610.01488/) (2026)

## Abstract (arXiv)

This paper presents our latest investigation on modeling backchannel in conversations. Motivated by a proactive backchanneling theory, we aim at developing a system which acts as a proactive listener by inserting backchannels, such as continuers and assessment, to influence speakers. Our model takes into account not only lexical and acoustic cues, but also introduces the simple and novel idea of using listener embeddings to mimic different backchanneling behaviours. Our experimental results on the Switchboard benchmark dataset reveal that acoustic cues are more important than lexical cues in this task and their combination with listener embeddings works best on both, manual transcriptions and automatically generated transcriptions.