# MUSE: Flexible Voiceprint Receptive Fields and Multi-Path Fusion Enhanced Taylor Transformer for U-Net-based Speech Enhancement

arXiv: https://arxiv.org/abs/2406.04589 · HTML: https://mlsanity.com/paper/2406.04589/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MB-TaylorFormer: Multi-branch Efficient Transformer Expanded by Taylor Formula for Image Dehazing](https://mlsanity.com/paper/2308.14036/) (2023)
- [An Empirical Study of Spatial Attention Mechanisms in Deep Networks](https://mlsanity.com/paper/1904.05873/) (2019)
- [Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed](https://mlsanity.com/paper/1909.01174/) (2019)
- [MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra](https://mlsanity.com/paper/2305.13686/) (2023)
- [PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network](https://mlsanity.com/paper/1911.04697/) (2019)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain](https://mlsanity.com/paper/2103.09963/) (2021)
- [Differentiable Consistency Constraints for Improved Deep Speech Enhancement](https://mlsanity.com/paper/1811.08521/) (2018)
- [DPT-FSNet: Dual-path Transformer Based Full-band and Sub-band Fusion Network for Speech Enhancement](https://mlsanity.com/paper/2104.13002/) (2021)
- [Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention](https://mlsanity.com/paper/2002.05873/) (2020)
- [MANNER: Multi-view Attention Network for Noise Erasure](https://mlsanity.com/paper/2203.02181/) (2022)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [A Two-Stage U-Net for High-Fidelity Denoising of Historical Recordings](https://mlsanity.com/paper/2202.08702/) (2022)
- [Multi-View Attention Transfer for Efficient Speech Enhancement](https://mlsanity.com/paper/2208.10367/) (2022)
- [Efficient Encoder-Decoder and Dual-Path Conformer for Comprehensive Feature Learning in Speech Enhancement](https://mlsanity.com/paper/2306.05861/) (2023)

## Built on by (most cited first, 4 of 4)

- [Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement](https://mlsanity.com/paper/2412.16626/) (2024)
- [Exploring Resolution-Wise Shared Attention in Hybrid Mamba-U-Nets for Improved Cross-Corpus Speech Enhancement](https://mlsanity.com/paper/2510.01958/) (2025)
- [IMSE: Efficient U-Net-based Speech Enhancement using Inception Depthwise Convolution and Amplitude-Aware Linear Attention](https://mlsanity.com/paper/2511.14515/) (2025)
- [BASENet: Band-Adapted Speech Enhancement Network with Cross-Band Attention](https://mlsanity.com/paper/2606.12662/) (2026)

## Abstract (arXiv)

Achieving a balance between lightweight design and high performance remains a challenging task for speech enhancement. In this paper, we introduce Multi-path Enhanced Taylor (MET) Transformer based U-net for Speech Enhancement (MUSE), a lightweight speech enhancement network built upon the Unet architecture. Our approach incorporates a novel Multi-path Enhanced Taylor (MET) Transformer block, which integrates Deformable Embedding (DE) to enable flexible receptive fields for voiceprints. The MET Transformer is uniquely designed to fuse Channel and Spatial Attention (CSA) branches, facilitating channel information exchange and addressing spatial attention deficits within the Taylor-Transformer framework. Through extensive experiments conducted on the VoiceBank+DEMAND dataset, we demonstrate that MUSE achieves competitive performance while significantly reducing both training and deployment costs, boasting a mere 0.51M parameters.