# Unified Target-Speaker ASR with Text and Enrollment Speech Cues

arXiv: https://arxiv.org/abs/2609.33853 · HTML: https://mlsanity.com/paper/2609.33853/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking](https://mlsanity.com/paper/2303.00332/) (2023)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking](https://mlsanity.com/paper/1810.04826/) (2018)
- [Adapting self-supervised models to multi-talker speech recognition using speaker embeddings](https://mlsanity.com/paper/2211.00482/) (2022)
- [Extending Whisper with prompt tuning to target-speaker ASR](https://mlsanity.com/paper/2312.08079/) (2023)
- [Conformer-based Target-Speaker Automatic Speech Recognition for Single-Channel Audio](https://mlsanity.com/paper/2308.05218/) (2023)
- [Streaming Target-Speaker ASR with Neural Transducer](https://mlsanity.com/paper/2209.04175/) (2022)
- [Target Speaker ASR with Whisper](https://mlsanity.com/paper/2409.09543/) (2024)
- [Target Speaker Extraction by Directly Exploiting Contextual Information in the Time-Frequency Domain](https://mlsanity.com/paper/2402.17146/) (2024)
- [USEF-TSE: Universal Speaker Embedding Free Target Speaker Extraction](https://mlsanity.com/paper/2409.02615/) (2024)
- [SQ-Whisper: Speaker-Querying based Whisper Model for Target-Speaker ASR](https://mlsanity.com/paper/2412.05589/) (2024)
- [Weakly-Supervised Speech Pre-training: A Case Study on Target Speech Recognition](https://mlsanity.com/paper/2305.16286/) (2023)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [Enhancing Intelligibility for Generative Target Speech Extraction via Joint Optimization with Target Speaker ASR](https://mlsanity.com/paper/2501.14477/) (2025)
- [SE-DiCoW: Self-Enrolled Diarization-Conditioned Whisper](https://mlsanity.com/paper/2601.19194/) (2026)
- [SEF-PNet: Speaker Encoder-Free Personalized Speech Enhancement with Local and Global Contexts Aggregation](https://mlsanity.com/paper/2501.11274/) (2025)
- [CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASR](https://mlsanity.com/paper/2601.22792/) (2026)
- [Detect, Attend and Extract: Keyword Guided Target Speaker Extraction](https://mlsanity.com/paper/2602.07977/) (2026)

## Abstract (arXiv)

Target-speaker automatic speech recognition (TS-ASR) aims to recognize a designated speaker while suppressing interfering speech in multi-talker environments. Conventional TS-ASR typically relies on an enrollment utterance, whereas text-guided methods use known lexical content, such as a wake word, to identify the target speaker from the observed mixture. These two cues provide complementary information but are usually studied separately. We propose a Unified Dual-Cue TS-ASR framework that supports text cues, enrollment speech, or both within a single model. Text cues interact with the mixture representation to extract target-speaker information conditioned on known lexical content, while an independent enrollment utterance provides complementary speaker information. Cross-attention cue-conditioning modules are integrated into shared Conformer blocks, and negative-cue sampling provides cue-validity supervision during dual-cue training. Experiments on 30,000 two-speaker mixtures across five recording/domain conditions and four oracle text-cue lengths show that, with five-character text cues, the concatenated dual-cue method achieves 8.80% CER, compared with 17.32% for text-only and 29.06% for enrollment-only inference. It also outperforms parallel dual-cue fusion (9.49% CER) and yields lower dual-cue CER across all five evaluation subsets. These results demonstrate the benefit of jointly exploiting complementary lexical and speaker information for target-speaker ASR.