# Cross-Attention End-to-End ASR for Two-Party Conversations

arXiv: https://arxiv.org/abs/1907.10726 · HTML: https://mlsanity.com/paper/1907.10726/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 14 of 14)

- [ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet](https://mlsanity.com/paper/2111.14706/) (2021)
- [Conversational Speech Recognition By Learning Conversation-level Characteristics](https://mlsanity.com/paper/2202.07855/) (2022)
- [Cross-utterance Reranking Models with BERT and Graph Convolutional Networks for Conversational Speech Recognition](https://mlsanity.com/paper/2106.06922/) (2021)
- [Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASR](https://mlsanity.com/paper/2207.01039/) (2022)
- [Context-aware Fine-tuning of Self-supervised Speech Models](https://mlsanity.com/paper/2212.08542/) (2022)
- [Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understanding](https://mlsanity.com/paper/2112.06743/) (2021)
- [BERT Attends the Conversation: Improving Low-Resource Conversational ASR](https://mlsanity.com/paper/2110.02267/) (2021)
- [Improving Transformer-based Conversational ASR by Inter-Sentential Attention Mechanism](https://mlsanity.com/paper/2207.00883/) (2022)
- [Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation](https://mlsanity.com/paper/2310.14278/) (2023)
- [Joint Modelling of Spoken Language Understanding Tasks with Integrated Dialog History](https://mlsanity.com/paper/2305.00926/) (2023)
- [Enhancing End-to-End Conversational Speech Translation Through Target Language Context Utilization](https://mlsanity.com/paper/2309.15686/) (2023)
- [Generative Context-aware Fine-tuning of Self-supervised Speech Models](https://mlsanity.com/paper/2312.09895/) (2023)
- [Cross-Attention is Half Explanation in Speech-to-Text Models](https://mlsanity.com/paper/2509.18010/) (2025)
- [Do You Listen with One or Two Microphones? A Unified ASR Model for Single and Multi-Channel Audio](https://mlsanity.com/paper/2106.02750/) (2021)

## Abstract (arXiv)

We present an end-to-end speech recognition model that learns interaction between two speakers based on the turn-changing information. Unlike conventional speech recognition models, our model exploits two speakers' history of conversational-context information that spans across multiple turns within an end-to-end framework. Specifically, we propose a speaker-specific cross-attention mechanism that can look at the output of the other speaker side as well as the one of the current speaker for better at recognizing long conversations. We evaluated the models on the Switchboard conversational speech corpus and show that our model outperforms standard end-to-end speech recognition models.