mlsanityData to 14 Oct 2026

Space-Time Memory Network for Sounding Object Localization in Videos

arXiv 2111.05526 ↗First posted 10 Nov 2021cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Leveraging temporal synchronization and association within sight and sound is an essential step towards robust localization of sounding objects. To this end, we propose a space-time memory network for sounding object localization in videos. It can simultaneously learn spatio-temporal attention over both uni-modal and cross-modal representations from audio and visual modalities. We show and analyze both quantitatively and qualitatively the effectiveness of incorporating spatio-temporal learning in localizing audio-visual objects. We demonstrate that our approach generalizes over various complex audio-visual scenes and outperforms recent state-of-the-art methods.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

25 of 35 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  5. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  6. 2018The Sound of Pixelscited by 259
  7. 2017Objects that Soundcited by 256
  8. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  9. 2018Learning to Localize Sound Source in Visual Scenescited by 189
  10. 2020Self-Supervised Learning of Audio-Visual Objects from Videocited by 188
  11. 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
  12. 2019The Sound of Motionscited by 146
13 more
  1. 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
  2. 2019SCSampler: Sampling Salient Clips from Video for Efficient Action Recognitioncited by 134
  3. 2020Multiple Sound Sources Localization from Coarse to Finecited by 118
  4. 2018Learning to Separate Object Sounds by Watching Unlabeled Videocited by 114
  5. 2020Music Gesture for Visual Sound Separationcited by 112
  6. 2020Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matchingcited by 104
  7. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 96
  8. 2019Self-supervised Moving Vehicle Tracking with Stereo Soundcited by 63
  9. 2021Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separationcited by 59
  10. 2019Watch, Listen and Tell: Multi-modal Weakly Supervised Dense Event Captioningcited by 38
  11. 2020Large Scale Audiovisual Learning of Sounds with Weakly Labeled Datacited by 27
  12. 2021Can audio-visual integration strengthen robustness under multimodal attacks?cited by 23
  13. 2018Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioningcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 2; 2026: 2.

10 papers.

  1. 2023Egocentric Audio-Visual Object Localizationcited by 40
  2. 2023Audio-Visual Class-Incremental Learningcited by 33
  3. 2023Sound Source Localization is All about Cross-Modal Alignmentcited by 31
  4. 2025Seeing Sound, Hearing Sight: Uncovering Modality Bias and Conflict of AI models in Sound Localizationcited by 18
  5. 2023Can CLIP Help Sound Source Localization?cited by 14
  6. 2024Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignmentcited by 8
  7. 2024Unveiling Visual Biases in Audio-Visual Localization Benchmarkscited by 3
  8. 2025Hearing and Seeing Through CLIP: A Framework for Self-Supervised Sound Source Localizationcited by 1
  9. 2026SOUPLE: Enhancing Audio-Visual Localization and Segmentation with Learnable Prompt Contextscited by 0
  10. 2026SceneBind: Binding What and Where Across Vision, Audio and Languagecited by 0