# Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

arXiv: https://arxiv.org/abs/2112.02321 · HTML: https://mlsanity.com/paper/2112.02321/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2021.

## Built on by (most cited first, 30 of 32)

- [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022)
- [An efficient encoder-decoder architecture with top-down attention for speech separation](https://mlsanity.com/paper/2209.15200/) (2022)
- [SepPrune: Structured Pruning for Efficient Deep Speech Separation](https://mlsanity.com/paper/2505.12079/) (2025)
- [SafeEar: Content Privacy-Preserving Audio Deepfake Detection](https://mlsanity.com/paper/2409.09272/) (2024)
- [SepIt: Approaching a Single Channel Speech Separation Bound](https://mlsanity.com/paper/2205.11801/) (2022)
- [PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-performance Cloud Removal from Multi-temporal Satellite Imagery](https://mlsanity.com/paper/2303.16565/) (2023)
- [IIANet: An Intra- and Inter-Modality Attention Network for Audio-Visual Speech Separation](https://mlsanity.com/paper/2308.08143/) (2023)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation](https://mlsanity.com/paper/2302.11131/) (2023)
- [An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits](https://mlsanity.com/paper/2212.10744/) (2022)
- [Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inference](https://mlsanity.com/paper/2312.03464/) (2023)
- [A Neural State-Space Model Approach to Efficient Speech Separation](https://mlsanity.com/paper/2305.16932/) (2023)
- [BASEN: Time-Domain Brain-Assisted Speech Enhancement Network with Convolutional Cross Attention in Multi-talker Conditions](https://mlsanity.com/paper/2305.09994/) (2023)
- [Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model](https://mlsanity.com/paper/2306.00160/) (2023)
- [SonicSim: A customizable simulation platform for speech processing in moving sound source scenarios](https://mlsanity.com/paper/2410.01481/) (2024)
- [Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks](https://mlsanity.com/paper/2203.16040/) (2022)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [CasNet: Investigating Channel Robustness for Speech Separation](https://mlsanity.com/paper/2210.15370/) (2022)
- [Efficient Encoder-Decoder and Dual-Path Conformer for Comprehensive Feature Learning in Speech Enhancement](https://mlsanity.com/paper/2306.05861/) (2023)
- [An Efficient Speech Separation Network Based on Recurrent Fusion Dilated Convolution and Channel Attention](https://mlsanity.com/paper/2306.05887/) (2023)
- [RTFS-Net: Recurrent Time-Frequency Modelling for Efficient Audio-Visual Speech Separation](https://mlsanity.com/paper/2309.17189/) (2023)
- [PanBench: Towards High-Resolution and High-Performance Pansharpening](https://mlsanity.com/paper/2311.12083/) (2023)
- [TDFNet: An Efficient Audio-Visual Speech Separation Model with Top-down Fusion](https://mlsanity.com/paper/2401.14185/) (2024)
- [SPMamba: State-space model is all you need in speech separation](https://mlsanity.com/paper/2404.02063/) (2024)
- [AV-CrossNet: an Audiovisual Complex Spectral Mapping Network for Speech Separation By Leveraging Narrow- and Cross-Band Modeling](https://mlsanity.com/paper/2406.11619/) (2024)
- [Papez: Resource-Efficient Speech Separation with Auditory Working Memory](https://mlsanity.com/paper/2407.00888/) (2024)
- [Time-Frequency-Based Attention Cache Memory Model for Real-Time Speech Separation](https://mlsanity.com/paper/2505.13094/) (2025)
- [Omni-directional attention mechanism based on Mamba for speech separation](https://mlsanity.com/paper/2601.16603/) (2026)
- [Difference-Driven Gating: Adaptive Feature Fusion for U-Net Decoder](https://mlsanity.com/paper/2607.11096/) (2026)
- [ARFA: An Asymmetric Receptive Field Autoencoder Model for Spatiotemporal Prediction](https://mlsanity.com/paper/2309.00314/) (2023)

## Abstract (arXiv)

Recent advances in the design of neural network architectures, in particular those specialized in modeling sequences, have provided significant improvements in speech separation performance. In this work, we propose to use a bio-inspired architecture called Fully Recurrent Convolutional Neural Network (FRCNN) to solve the separation task. This model contains bottom-up, top-down and lateral connections to fuse information processed at various time-scales represented by textitstages. In contrast to the traditional approach updating stages in parallel, we propose to first update the stages one by one in the bottom-up direction, then fuse information from adjacent stages simultaneously and finally fuse information from all stages to the bottom stage together. Experiments showed that this asynchronous updating scheme achieved significantly better results with much fewer parameters than the traditional synchronous updating scheme. In addition, the proposed model achieved good balance between speech separation accuracy and computational efficiency as compared to other state-of-the-art models on three benchmark datasets.