mlsanityData to 19 Oct 2026

End-to-End Target Speaker Speech Recognition Using Context-Aware Attention Mechanisms for Challenging Enrollment Scenario

arXiv 2501.15466 ↗First posted 26 Jan 2025eess.AS cs.SDcited by 0

On the map: Speech and audio language models → Speaker recognition and diarization

Abstract (arXiv)

This paper presents a novel streaming end-to-end target-speaker speech recognition that addresses two critical limitations in systems: the handling of noisy enrollment utterances and specific enrollment phrase requirements. This paper proposes a robust Target-Speaker Recurrent Neural Network Transducer (TS-RNNT) with dual attention mechanisms for contextual biasing and overlapping enrollment processing. The model incorporates a text decoder and attention mechanism specifically designed to extract relevant speaker characteristics from noisy, overlapping enrollment audio. Experimental results on a synthesized dataset demonstrate the model's resilience, maintaining a Word Error Rate (WER) of 16.44% even with overlapping enrollment at 5dB Signal-to-Interference Ratio (SIR), compared to conventional approaches that degrade to WERs above 75% under similar conditions. This significant performance improvement, coupled with the model's semi-text-dependent enrollment capabilities, represents a substantial advancement toward more practical and versatile voice-controlled devices.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models. Builds on 13 of its 32 references that are in the map. First posted on arXiv 26 Jan 2025.

Builds on

13 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Streaming Target-Speaker ASR with Neural Transducer9 Sep

Other references · 12

  1. 2017Attention Is All You Needcited by 61,180
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 2,293
  3. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 717
  4. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 246
  5. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 122
  6. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 112
  7. 2020Listen to What You Want: Neural Network-based Universal Sound Selectorcited by 32
  8. 2020End-to-End Neural Diarization: Reformulating Speaker Diarization as Simple Multi-label Classificationcited by 24
  9. 2022Real-Time Target Sound Extractioncited by 24
  10. 2020Improving RNN Transducer With Target Speaker Extraction and Neural Uncertainty Estimationcited by 7
  11. 2019End-to-end Anchored Speech Recognitioncited by 5
  12. 2022Anchored Speech Recognition with Neural Transducerscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unified Target-Speaker ASR with Text and Enrollment Speech Cues
  2. 2022Contextual-Utterance Training for Automatic Speech Recognition
  3. 2021Multi-turn RNN-T for streaming recognition of multi-party speech
  4. 2023Improving Scheduled Sampling for Neural Transducer-based ASR
  5. 2022Joint Speaker Encoder and Neural Back-end Model for Fully End-to-End Automatic Speaker Verification with Multiple Enrollment Utterances
  6. 2023Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data
  7. 2021Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
  8. 2021Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers