# TF-Locoformer: Transformer with Local Modeling by Convolution for Speech Separation and Enhancement

arXiv: https://arxiv.org/abs/2408.03440 · HTML: https://mlsanity.com/paper/2408.03440/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2024.

## Built on by (most cited first, 11 of 11)

- [Source Separation by Flow Matching](https://mlsanity.com/paper/2505.16119/) (2025)
- [A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy References](https://mlsanity.com/paper/2508.14623/) (2025)
- [Knowing When to Quit: Probabilistic Early Exits for Speech Separation](https://mlsanity.com/paper/2507.09768/) (2025)
- [U-DREAM: Unsupervised Dereverberation guided by a Reverberation Model](https://mlsanity.com/paper/2507.14237/) (2025)
- [TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation](https://mlsanity.com/paper/2509.15666/) (2025)
- [Unsupervised Single-Channel Speech Separation with Diffusion under Speaker-Embedding Guidance](https://mlsanity.com/paper/2509.24395/) (2025)
- [The silence of the weights: a structural pruning strategy for attention-based audio signal architectures with second order metrics](https://mlsanity.com/paper/2509.26207/) (2025)
- [Source Separation for A Cappella Music](https://mlsanity.com/paper/2509.26580/) (2025)
- [Is Phase Really Needed for Weakly-Supervised Dereverberation ?](https://mlsanity.com/paper/2511.17346/) (2025)
- [SURF: Separation via Unsupervised Remixing Flow](https://mlsanity.com/paper/2606.04921/) (2026)
- [DTT-BSR+: A Generative-Regression Cascade for Music Source Restoration](https://mlsanity.com/paper/2606.24127/) (2026)

## Abstract (arXiv)

Time-frequency (TF) domain dual-path models achieve high-fidelity speech separation. While some previous state-of-the-art (SoTA) models rely on RNNs, this reliance means they lack the parallelizability, scalability, and versatility of Transformer blocks. Given the wide-ranging success of pure Transformer-based architectures in other fields, in this work we focus on removing the RNN from TF-domain dual-path models, while maintaining SoTA performance. This work presents TF-Locoformer, a Transformer-based model with LOcal-modeling by COnvolution. The model uses feed-forward networks (FFNs) with convolution layers, instead of linear layers, to capture local information, letting the self-attention focus on capturing global patterns. We place two such FFNs before and after self-attention to enhance the local-modeling capability. We also introduce a novel normalization for TF-domain dual-path models. Experiments on separation and enhancement datasets show that the proposed model meets or exceeds SoTA in multiple benchmarks with an RNN-free architecture.