# Position-Aware Target Speaker Extraction for Long-Form Multi-Party Conversations: A Diarization-Free Framework for ASR

arXiv: https://arxiv.org/abs/2606.29497 · HTML: https://mlsanity.com/paper/2606.29497/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 25 of 25)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [End-to-End Neural Speaker Diarization with Permutation-Free Objectives](https://mlsanity.com/paper/1909.05952/) (2019)
- [Continuous speech separation: dataset and analysis](https://mlsanity.com/paper/2001.11482/) (2020)
- [Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis](https://mlsanity.com/paper/2011.02014/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration](https://mlsanity.com/paper/1810.11359/) (2018)
- [GPU-accelerated Guided Source Separation for Meeting Transcription](https://mlsanity.com/paper/2212.05271/) (2022)
- [TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings](https://mlsanity.com/paper/2303.03849/) (2023)
- [Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems](https://mlsanity.com/paper/2409.06656/) (2024)
- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio](https://mlsanity.com/paper/2511.16046/) (2025)
- [USEV: Universal Speaker Extraction with Visual Cue](https://mlsanity.com/paper/2109.14831/) (2021)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [Graph-PIT: Generalized permutation invariant training for continuous separation of arbitrary numbers of speakers](https://mlsanity.com/paper/2107.14446/) (2021)
- [CASA-ASR: Context-Aware Speaker-Attributed ASR](https://mlsanity.com/paper/2305.12459/) (2023)
- [Speaker activity driven neural speech extraction](https://mlsanity.com/paper/2101.05516/) (2021)
- [Low-Latency Speaker-Independent Continuous Speech Separation](https://mlsanity.com/paper/1904.06478/) (2019)
- [Exploiting spatial information with the informed complex-valued spatial autoencoder for target speaker extraction](https://mlsanity.com/paper/2210.15512/) (2022)
- [Multi-channel Conversational Speaker Separation via Neural Diarization](https://mlsanity.com/paper/2311.08630/) (2023)
- [DCF-DS: Deep Cascade Fusion of Diarization and Separation for Speech Recognition under Realistic Single-Channel Conditions](https://mlsanity.com/paper/2411.06667/) (2024)
- [Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systems](https://mlsanity.com/paper/2507.07518/) (2025)
- [Beamformer-Guided Target Speaker Extraction](https://mlsanity.com/paper/2303.08702/) (2023)
- [Multi-resolution location-based training for multi-channel continuous speech separation](https://mlsanity.com/paper/2301.06458/) (2023)

## Abstract (arXiv)

In long-form multi-party conversations, highly imbalanced speaker activity and frequent overlap make it difficult to identify "who spoke when and what". Sliding-window continuous speech separation (CSS) mitigates sparse supervision, but often suffers from cross-window speaker inconsistency and residual crosstalk, which in practice requires diarization for reliable speaker attribution. Motivated by the stability of speakers' directions of arrival (DOAs) in meetings, we propose PATSE, a multi-channel Position-Aware Target Speaker Extraction front-end that uses DOA as a spatial prior to directly extract the speech of each target speaker. PATSE combines a DOA-guided spatial encoder and conditioner to generate speaker-attributed streams, from which speaker activity can be inferred via simple post-processing (e.g., VAD) without explicit diarization. Experiments on both replayed and real conversations show consistent ASR gains outperforming CSS and diarization-based pipelines.