# MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation

arXiv: https://arxiv.org/abs/2312.11825 · HTML: https://mlsanity.com/paper/2312.11825/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2023.

## Built on by (most cited first, 30 of 39)

- [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025)
- [Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation](https://mlsanity.com/paper/2403.18257/) (2024)
- [MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection](https://mlsanity.com/paper/2403.19888/) (2024)
- [INFP: Audio-Driven Interactive Head Generation in Dyadic Conversations](https://mlsanity.com/paper/2412.04037/) (2024)
- [LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement](https://mlsanity.com/paper/2503.00493/) (2025)
- [MOSS-TTS Technical Report](https://mlsanity.com/paper/2603.18090/) (2026)
- [Towards Audio Codec-based Speech Separation](https://mlsanity.com/paper/2406.12434/) (2024)
- [SepMamba: State-space models for speaker separation using Mamba](https://mlsanity.com/paper/2410.20997/) (2024)
- [OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions](https://mlsanity.com/paper/2505.21724/) (2025)
- [LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization](https://mlsanity.com/paper/2409.00819/) (2024)
- [SonicSim: A customizable simulation platform for speech processing in moving sound source scenarios](https://mlsanity.com/paper/2410.01481/) (2024)
- [The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition](https://mlsanity.com/paper/2505.13971/) (2025)
- [UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement](https://mlsanity.com/paper/2510.20441/) (2025)
- [SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation](https://mlsanity.com/paper/2505.03273/) (2025)
- [A Semantic Information-based Hierarchical Speech Enhancement Method Using Factorized Codec and Diffusion Model](https://mlsanity.com/paper/2505.13843/) (2025)
- [UmbraTTS: Adapting Text-to-Speech to Environmental Contexts with Flow Matching](https://mlsanity.com/paper/2506.09874/) (2025)
- [Knowing When to Quit: Probabilistic Early Exits for Speech Separation](https://mlsanity.com/paper/2507.09768/) (2025)
- [MOSS-TTSD: Text to Spoken Dialogue Generation](https://mlsanity.com/paper/2603.19739/) (2026)
- [Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models](https://mlsanity.com/paper/2609.22452/) (2026)
- [Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better](https://mlsanity.com/paper/2409.18971/) (2024)
- [WESR: Scaling and Evaluating Word-level Event-Speech Recognition](https://mlsanity.com/paper/2601.04508/) (2026)
- [SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation](https://mlsanity.com/paper/2601.19702/) (2026)
- [MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR](https://mlsanity.com/paper/2603.10465/) (2026)
- [TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech](https://mlsanity.com/paper/2606.30543/) (2026)
- [AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing](https://mlsanity.com/paper/2609.08936/) (2026)
- [Effects of Dataset Sampling Rate for Noise Cancellation through Deep Learning](https://mlsanity.com/paper/2405.20884/) (2024)
- [Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speech](https://mlsanity.com/paper/2507.13563/) (2025)
- [TFGA-Net: Temporal-Frequency Graph Attention Network for Brain-Controlled Speaker Extraction](https://mlsanity.com/paper/2510.12275/) (2025)
- [Speech Separation for Hearing-Impaired Children in the Classroom](https://mlsanity.com/paper/2511.07677/) (2025)
- [Lightweight Wasserstein Audio-Visual Model for Unified Speech Enhancement and Separation](https://mlsanity.com/paper/2512.06689/) (2025)

## Abstract (arXiv)

Our previously proposed MossFormer has achieved promising performance in monaural speech separation. However, it predominantly adopts a self-attention-based MossFormer module, which tends to emphasize longer-range, coarser-scale dependencies, with a deficiency in effectively modelling finer-scale recurrent patterns. In this paper, we introduce a novel hybrid model that provides the capabilities to model both long-range, coarse-scale dependencies and fine-scale recurrent patterns by integrating a recurrent module into the MossFormer framework. Instead of applying the recurrent neural networks (RNNs) that use traditional recurrent connections, we present a recurrent module based on a feedforward sequential memory network (FSMN), which is considered "RNN-free" recurrent network due to the ability to capture recurrent patterns without using recurrent connections. Our recurrent module mainly comprises an enhanced dilated FSMN block by using gated convolutional units (GCU) and dense connections. In addition, a bottleneck layer and an output layer are also added for controlling information flow. The recurrent module relies on linear projections and convolutions for seamless, parallel processing of the entire sequence. The integrated MossFormer2 hybrid model demonstrates remarkable enhancements over MossFormer and surpasses other state-of-the-art methods in WSJ0-2/3mix, Libri2Mix, and WHAM!/WHAMR! benchmarks (https://github.com/modelscope/ClearerVoice-Studio).