mlsanityData to 19 Oct 2026

Position-Aware Target Speaker Extraction for Long-Form Multi-Party Conversations: A Diarization-Free Framework for ASR

arXiv 2606.29497 ↗First posted 28 Jun 2026cs.SD cs.MMcited by 0

On the map: Speech and audio language models → Speaker recognition and diarization

Abstract (arXiv)

In long-form multi-party conversations, highly imbalanced speaker activity and frequent overlap make it difficult to identify "who spoke when and what". Sliding-window continuous speech separation (CSS) mitigates sparse supervision, but often suffers from cross-window speaker inconsistency and residual crosstalk, which in practice requires diarization for reliable speaker attribution. Motivated by the stability of speakers' directions of arrival (DOAs) in meetings, we propose PATSE, a multi-channel Position-Aware Target Speaker Extraction front-end that uses DOA as a spatial prior to directly extract the speech of each target speaker. PATSE combines a DOA-guided spatial encoder and conditioner to generate speaker-attributed streams, from which speaker activity can be inferred via simple post-processing (e.g., VAD) without explicit diarization. Experiments on both replayed and real conversations show consistent ASR gains outperforming CSS and diarization-based pipelines.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models. Builds on 25 of its 35 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

25 of 35 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio20 Nov
  2. 2024DCF-DS: Deep Cascade Fusion of Diarization and Separation for Speech Recognition under Realistic Single-Channel Conditions11 Nov
  3. 2024Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems10 Sep
  4. 2023Multi-channel Conversational Speaker Separation via Neural Diarization15 Nov
  5. 2023One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition2 Oct
  6. 2023CASA-ASR: Context-Aware Speaker-Attributed ASR21 May
  7. 2023Multi-resolution location-based training for multi-channel continuous speech separation16 Jan

Other references · 18

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,420
  3. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 230
  4. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 158
  5. 2020Continuous speech separation: dataset and analysiscited by 124
  6. 2020Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysiscited by 71
  7. 2019End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separationcited by 70
  8. 2018gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Accelerationcited by 54
  9. 2022GPU-accelerated Guided Source Separation for Meeting Transcriptioncited by 33
  10. 2023TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddingscited by 27
  11. 2021USEV: Universal Speaker Extraction with Visual Cuecited by 17
  12. 2024TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separationcited by 16
6 more
  1. 2021Graph-PIT: Generalized permutation invariant training for continuous separation of arbitrary numbers of speakerscited by 14
  2. 2021Speaker activity driven neural speech extractioncited by 11
  3. 2019Low-Latency Speaker-Independent Continuous Speech Separationcited by 9
  4. 2022Exploiting spatial information with the informed complex-valued spatial autoencoder for target speaker extractioncited by 8
  5. 2025Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systemscited by 5
  6. 2023Beamformer-Guided Target Speaker Extractioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Listening to Multi-talker Conversations: Modular and End-to-end Perspectives
  2. 2026DM-ASR: Diarization-aware Multi-speaker ASR with Large Language Models
  3. 2025Diarization-Aware Multi-Speaker Automatic Speech Recognition via Large Language Models
  4. 2021Localization Based Sequential Grouping for Continuous Speech Separation
  5. 2025Scaling Multi-Talker ASR with Speaker-Agnostic Activity Streams
  6. 2026Who Spoke When in Multi-Conversation: Target Speaker Tagging Task and Benchmark
  7. 2026Pushing the Boundaries of Streaming Multi-Speaker ASR: A Systematic Study of Architectural Trade-offs
  8. 2021A Real-time Speaker Diarization System Based on Spatial Spectrum