mlsanityData to 7 Oct 2026

End-to-End Joint Target and Non-Target Speakers ASR

arXiv 2306.02273 ↗First posted 4 Jun 2023cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel automatic speech recognition (ASR) system that can transcribe individual speaker's speech while identifying whether they are target or non-target speakers from multi-talker overlapped speech. Target-speaker ASR systems are a promising way to only transcribe a target speaker's speech by enrolling the target speaker's information. However, in conversational ASR applications, transcribing both the target speaker's speech and non-target speakers' ones is often required to understand interactive information. To naturally consider both target and non-target speakers in a single ASR model, our idea is to extend autoregressive modeling-based multi-talker ASR systems to utilize the enrollment speech of the target speaker. Our proposed ASR is performed by recursively generating both textual tokens and tokens that represent target or non-target speakers. Our experiments demonstrate the effectiveness of our proposed method.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

24 of 32 references.

References · 24

  1. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  2. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  3. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  4. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
  5. 2020Serialized Output Training for End-to-End Overlapped Speech Recognitioncited by 87
  6. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 72
  7. 2020Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenariocited by 67
  8. 2018ArcFace: Additive Angular Margin Loss for Deep Face Recognitioncited by 56
  9. 2020End-to-End Multi-speaker Speech Recognition with Transformercited by 42
  10. 2017Recognizing Multi-talker Speech with Permutation Invariant Trainingcited by 33
  11. 2023Neural Target Speech Extraction: An Overviewcited by 33
  12. 2018A Purely End-to-end System for Multi-speaker Speech Recognitioncited by 30
12 more
  1. 2020VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognitioncited by 29
  2. 2018End-to-End Monaural Multi-speaker ASR System without Pretrainingcited by 26
  3. 2019Personal VAD: Speaker-Conditioned Voice Activity Detectioncited by 24
  4. 2020Streaming Multi-speaker ASR with RNN-Tcited by 22
  5. 2019Auxiliary Interference Speaker Loss for Target-Speaker Speech Recognitioncited by 16
  6. 2021Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speakercited by 15
  7. 2022Personal VAD 2.0: Optimizing Personal Voice Activity Detection for On-Device Speech Recognitioncited by 12
  8. 2019End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learningcited by 9
  9. 2022Streaming Target-Speaker ASR with Neural Transducercited by 8
  10. 2020Time-domain speaker extraction networkcited by 4
  11. 2021Enrollment-less training for personalized voice activity detectioncited by 4
  12. 2021Unified Autoregressive Modeling for Joint End-to-End Multi-Talker Overlapped Speech Recognition and Speaker Attribute Estimationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructionscited by 17
  2. 2024Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition Systemcited by 11
  3. 2024Serialized Output Training by Learned Dominancecited by 5
  4. 2026CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASRcited by 0