# DGFNet: End-to-End Audio-Visual Source Separation Based on Dynamic Gating Fusion

arXiv: https://arxiv.org/abs/2504.21366 · HTML: https://mlsanity.com/paper/2504.21366/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 5 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 3 of 3)

- [Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation](https://mlsanity.com/paper/1804.03619/) (2018)
- [Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning](https://mlsanity.com/paper/2008.05789/) (2020)
- [Visually Guided Sound Source Separation and Localization using Self-Supervised Motion Representations](https://mlsanity.com/paper/2104.08506/) (2021)

## Built on by (most cited first, 13 of 13)

- [Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigation](https://mlsanity.com/paper/2509.16924/) (2025)
- [Audio Spatially-Guided Fusion for Audio-Visual Navigation](https://mlsanity.com/paper/2604.02389/) (2026)
- [Spatial-Aware Conditioned Fusion for Audio-Visual Navigation](https://mlsanity.com/paper/2604.02390/) (2026)
- [Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation](https://mlsanity.com/paper/2604.02391/) (2026)
- [Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction](https://mlsanity.com/paper/2604.05007/) (2026)
- [EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence](https://mlsanity.com/paper/2604.23325/) (2026)
- [MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2607.10599/) (2026)
- [HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models](https://mlsanity.com/paper/2607.13072/) (2026)
- [CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigation](https://mlsanity.com/paper/2609.17420/) (2026)
- [Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigation](https://mlsanity.com/paper/2609.17421/) (2026)
- [Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splatting](https://mlsanity.com/paper/2609.17422/) (2026)
- [WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Field](https://mlsanity.com/paper/2609.21391/) (2026)
- [Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation](https://mlsanity.com/paper/2610.14077/) (2026)

## Abstract (arXiv)

Current Audio-Visual Source Separation methods primarily adopt two design strategies. The first strategy involves fusing audio and visual features at the bottleneck layer of the encoder, followed by processing the fused features through the decoder. However, when there is a significant disparity between the two modalities, this approach may lead to the loss of critical information. The second strategy avoids direct fusion and instead relies on the decoder to handle the interaction between audio and visual features. Nonetheless, if the encoder fails to integrate information across modalities adequately, the decoder may be unable to effectively capture the complex relationships between them. To address these issues, this paper proposes a dynamic fusion method based on a gating mechanism that dynamically adjusts the modality fusion degree. This approach mitigates the limitations of solely relying on the decoder and facilitates efficient collaboration between audio and visual features. Additionally, an audio attention module is introduced to enhance the expressive capacity of audio features, thereby further improving model performance. Experimental results demonstrate that our method achieves significant performance improvements on two benchmark datasets, validating its effectiveness and advantages in Audio-Visual Source Separation tasks.