mlsanityData to 19 Oct 2026

DBMIF: a deep balanced multimodal iterative fusion framework for air- and bone-conduction speech enhancement

arXiv 2603.02877 ↗First posted 3 Mar 2026eess.AScited by 0

On the map: Speech and audio language models → Speech enhancement and noise suppression

Abstract (arXiv)

The performance of conventional speech enhancement systems degrades sharply in extremely low signal-to-noise ratio (SNR) environments where air-conduction (AC) microphones are overwhelmed by ambient noise. Although bone-conduction (BC) sensors offer complementary, noise-tolerant information, existing fusion approaches struggle to maintain consistent performance across a wide range of SNR conditions. To address this limitation, we propose the Deep Balanced Multimodal Iterative Fusion Framework (DBMIF), a three-branch architecture designed to reconstruct high-fidelity speech through rigorous cross-modal interaction. Specifically, grounded in a multi-scale interactive encoder-decoder backbone, the framework orchestrates an iterative attention module and a cross-branch gated module to facilitate adaptive weighting and bidirectional exchange. To complement this dynamic interaction, a balanced-interaction bottleneck is further integrated to learn a compact, stable fused representation. Extensive experiments demonstrate that DBMIF achieves competitive performance compared with recent unimodal and multimodal baselines in both speech quality and intelligibility across diverse noise types. In downstream ASR tasks, the proposed method reduces the character error rate by at least 2.5 percent compared to competing approaches. These results confirm that DBMIF effectively harnesses the robustness of BC speech while preserving the naturalness of AC speech, ensuring reliability in real-world scenarios. The source code is publicly available at github.com/wyl516w/dbmif.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 19 of its 56 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

19 of 56 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Generic Speech Enhancement with Self-Supervised Representation Space Loss10 Jul
  2. 2021Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement22 Jun

Other references · 17

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  2. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 436
  3. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 356
  4. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 342
  5. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 284
  6. 2020Real Time Speech Enhancement in the Waveform Domaincited by 282
  7. 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 262
  8. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 177
  9. 2023ICASSP 2023 Deep Noise Suppression Challengecited by 101
  10. 2019Time-Domain Multi-modal Bone/air Conducted Speech Enhancementcited by 10
  11. 2022EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphonescited by 8
  12. 2023Leveraging Self-Supervised Audio-Visual Pretrained Models to Improve Vocoded Speech Intelligibility in Cochlear Implant Simulationcited by 7
5 more
  1. 2025LipGen: Viseme-Guided Lip Video Generation for Enhancing Visual Speech Recognitioncited by 6
  2. 2023Deep Equilibrium Multimodal Fusioncited by 5
  3. 2025AVE Speech: A Comprehensive Multi-Modal Dataset for Speech Recognition Integrating Audio, Visual, and Electromyographic Signalscited by 3
  4. 2021CycleGAN with Dual Adversarial Loss for Bone-Conducted Speech Enhancementcited by 1
  5. 2025Modality-Specific Speech Enhancement and Noise-Adaptive Fusion for Acoustic and Body-Conduction Microphone Frameworkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Bone-conduction Guided Multimodal Speech Enhancement with Conditional Diffusion Models
  2. 2025VibOmni: Towards Scalable Bone-conduction Speech Enhancement on Earables
  3. 2026Audio-Anchored Fusion of Multi-Ratio DiT Reconstruction Residuals for Cross-Domain Audio Deepfake Detection
  4. 2024TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms
  5. 2026DoubleHelix: Structured Cross-Modal Fusion for Audio-Visual Speech Recognition with LLMs
  6. 2026SLICE: Speech Enhancement via Layer-wise Injection of Conditioning Embeddings
  7. 2025FUSE: Universal Speech Enhancement using Multi-Stage Fusion of Sparse Compression and Token Generation Models for the URGENT 2025 Challenge
  8. 2022Optimizing Shoulder to Shoulder: A Coordinated Sub-Band Fusion Model for Real-Time Full-Band Speech Enhancement