# Continuous Speech Separation with Recurrent Selective Attention Network

arXiv: https://arxiv.org/abs/2110.14838 · HTML: https://mlsanity.com/paper/2110.14838/

## Key facts

- Cited by 2 papers in the map (data through 19 Oct 2026).
- First posted on arXiv 28 Oct 2021.

## Built on by (most cited first, 2 of 2)

- [SURT 2.0: Advances in Transducer-based Multi-talker Speech Recognition](https://mlsanity.com/paper/2306.10559/) (2023)
- [Multi-resolution location-based training for multi-channel continuous speech separation](https://mlsanity.com/paper/2301.06458/) (2023)

## Abstract (arXiv)

While permutation invariant training (PIT) based continuous speech separation (CSS) significantly improves the conversation transcription accuracy, it often suffers from speech leakages and failures in separation at "hot spot" regions because it has a fixed number of output channels. In this paper, we propose to apply recurrent selective attention network (RSAN) to CSS, which generates a variable number of output channels based on active speaker counting. In addition, we propose a novel block-wise dependency extension of RSAN by introducing dependencies between adjacent processing blocks in the CSS framework. It enables the network to utilize the separation results from the previous blocks to facilitate the current block processing. Experimental results on the LibriCSS dataset show that the RSAN-based CSS (RSAN-CSS) network consistently improves the speech recognition accuracy over PIT-based models. The proposed block-wise dependency modeling further boosts the performance of RSAN-CSS.