# FlowGrad: Using Motion for Visual Sound Source Localization

arXiv: https://arxiv.org/abs/2211.08367 · HTML: https://mlsanity.com/paper/2211.08367/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Objects that Sound](https://mlsanity.com/paper/1712.06651/) (2017)
- [Learning to Localize Sound Source in Visual Scenes](https://mlsanity.com/paper/1803.03849/) (2018)
- [Self-Supervised Learning of Audio-Visual Objects from Video](https://mlsanity.com/paper/2008.04237/) (2020)
- [Localizing Visual Sounds the Hard Way](https://mlsanity.com/paper/2104.02691/) (2021)
- [The Sound of Motions](https://mlsanity.com/paper/1904.05979/) (2019)
- [You Only Learn One Representation: Unified Network for Multiple Tasks](https://mlsanity.com/paper/2105.04206/) (2021)
- [Do We Need Sound for Sound Source Localization?](https://mlsanity.com/paper/2007.05722/) (2020)
- [How to Listen? Rethinking Visual Sound Localization](https://mlsanity.com/paper/2204.05156/) (2022)

## Built on by (most cited first, 4 of 4)

- [Audio-Visual Segmentation via Unlabeled Frame Exploitation](https://mlsanity.com/paper/2403.11074/) (2024)
- [Learning to Visually Localize Sound Sources from Mixtures without Prior Source Knowledge](https://mlsanity.com/paper/2403.17420/) (2024)
- [Attention-Map Augmentation for Hypercomplex Breast Cancer Classification](https://mlsanity.com/paper/2310.07633/) (2023)
- [Object-aware Sound Source Localization via Audio-Visual Scene Understanding](https://mlsanity.com/paper/2506.18557/) (2025)

## Abstract (arXiv)

Most recent work in visual sound source localization relies on semantic audio-visual representations learned in a self-supervised manner, and by design excludes temporal information present in videos. While it proves to be effective for widely used benchmark datasets, the method falls short for challenging scenarios like urban traffic. This work introduces temporal context into the state-of-the-art methods for sound source localization in urban scenes using optical flow as a means to encode motion information. An analysis of the strengths and weaknesses of our methods helps us better understand the problem of visual sound source localization and sheds light on open challenges for audio-visual scene understanding.