Heterogeneous Space Fusion and Dual-Dimension Attention: A New Paradigm for Speech Enhancement
Not on the map. Why?
Abstract (arXiv)
Self-supervised learning has demonstrated impressive performance in speech tasks, yet there remains ample opportunity for advancement in the realm of speech enhancement research. In addressing speech tasks, confining the attention mechanism solely to the temporal dimension poses limitations in effectively focusing on critical speech features. Considering the aforementioned issues, our study introduces a novel speech enhancement framework, HFSDA, which skillfully integrates heterogeneous spatial features and incorporates a dual-dimension attention mechanism to significantly enhance speech clarity and quality in noisy environments. By leveraging self-supervised learning embeddings in tandem with Short-Time Fourier Transform (STFT) spectrogram features, our model excels at capturing both high-level semantic information and detailed spectral data, enabling a more thorough analysis and refinement of speech signals. Furthermore, we employ the innovative Omni-dimensional Dynamic Convolution (ODConv) technology within the spectrogram input branch, enabling enhanced extraction and integration of crucial information across multiple dimensions. Additionally, we refine the Conformer model by enhancing its feature extraction capabilities not only in the temporal dimension but also across the spectral domain. Extensive experiments on the VCTK-DEMAND dataset show that HFSDA is comparable to existing state-of-the-art models, confirming the validity of our approach.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 13 Aug 2024.
Builds on
20 of 34 references.
References · 20
- 2017Attention Is All You Needcited by 60,361
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 141
- 2022Omni-Dimensional Dynamic Convolutioncited by 55
- 2022Investigating self-supervised learning for speech enhancement and separationcited by 35
- 2019T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancementcited by 34
- 2022MANNER: Multi-view Attention Network for Noise Erasurecited by 28
- 2023DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancementcited by 25
8 more
- 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
- 2022CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancementcited by 23
- 2020Dense CNN with Self-Attention for Time-Domain Speech Enhancementcited by 16
- 2020Self-Supervised learning with cross-modal transformers for emotion recognitioncited by 12
- 2022Exploring WavLM on Speech Enhancementcited by 9
- 2024MFHCA: Enhancing Speech Emotion Recognition Via Multi-Spatial Fusion and Hierarchical Cooperative Attentioncited by 6
- 2022Rethinking Gradient Operator for Exposing AI-enabled Face Forgeriescited by 5
- 2020A Recursive Network with Dynamic Attention for Monaural Speech Enhancementcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.