mlsanityData to 7 Oct 2026

Multiple Sound Sources Localization from Coarse to Fine

arXiv 2007.06355 ↗First posted 13 Jul 2020cs.CVcited by 118

Not on the map. Why?

Abstract (arXiv)

How to visually localize multiple sound sources in unconstrained videos is a formidable problem, especially when lack of the pairwise sound-object annotations. To solve this problem, we develop a two-stage audiovisual learning framework that disentangles audio and visual representations of different categories from complex scenes, then performs cross-modal feature alignment in a coarse-to-fine manner. Our model achieves state-of-the-art results on public dataset of localization, as well as considerable performance on multi-source sound localization in complex scenes. We then employ the localization results for sound separation and obtain comparable performance to existing methods. These outcomes demonstrate our model's ability in effectively aligning sounds with specific visual sources. Code is available at https://github.com/shvdiwnkozbw/Multi-Source-Sound-Localization

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 21; 2023: 34; 2024: 27; 2025: 18; 2026: 3.

118 papers.

Sort:

By area:

  1. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  2. 2021Localizing Visual Sounds the Hard Waycited by 169
  3. 2022Audio-Visual Segmentationcited by 148
  4. 2022Localizing Visual Sounds the Easy Waycited by 81
  5. 2023Audio-Visual Segmentation with Semanticscited by 73
  6. 2022A Closer Look at Weakly-Supervised Audio-Visual Source Localizationcited by 72
  7. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72
  8. 2021Positive Sample Propagation along the Audio-Visual Event Linecited by 69
  9. 2023AVSegFormer: Audio-Visual Segmentation with Transformercited by 66
  10. 2021Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separationcited by 59
  11. 2022Mix and Localize: Localizing Sound Sources in Mixturescited by 54
  12. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
38 more
  1. 2023Audio-Visual Grouping Network for Sound Localization from Mixturescited by 52
  2. 2023CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentationcited by 52
  3. 2022Contrastive Positive Sample Propagation along the Audio-Visual Event Linecited by 50
  4. 2021Visually Informed Binaural Audio Generation without Binaural Audioscited by 46
  5. 2023Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learningcited by 46
  6. 2021There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledgecited by 44
  7. 2023Annotation-free Audio-Visual Segmentationcited by 44
  8. 2022Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisationcited by 43
  9. 2023Egocentric Audio-Visual Object Localizationcited by 40
  10. 2021Audio-Visual Synchronisation in the wildcited by 39
  11. 2023Multimodal Variational Auto-encoder based Audio-Visual Segmentationcited by 37
  12. 2023Connecting Multi-modal Contrastive Representationscited by 36
  13. 2023Improving Audio-Visual Segmentation with Bidirectional Generationcited by 36
  14. 2021Class-aware Sounding Objects Localization via Audiovisual Correspondencecited by 32
  15. 2023Audio-aware Query-enhanced Transformer for Audio-Visual Segmentationcited by 32
  16. 2023Audio-Visual Segmentation by Exploring Cross-Modal Mutual Semanticscited by 32
  17. 2022Self-Supervised Predictive Learning: A Negative-Free Method for Sound Source Localization in Visual Scenescited by 31
  18. 2023Sound Source Localization is All about Cross-Modal Alignmentcited by 30
  19. 2021Self-supervised object detection from audio-visual correspondencecited by 29
  20. 2022iQuery: Instruments as Queries for Audio-Visual Sound Separationcited by 29
  21. 2021Unsupervised Sound Localization via Iterative Contrastive Learningcited by 28
  22. 2022AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localizationcited by 28
  23. 2022Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localizationcited by 27
  24. 2022Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Languagecited by 27
  25. 2023Discovering Sounding Objects by Audio Queries for Audio Visual Segmentationcited by 27
  26. 2024Audio-Visual Segmentation via Unlabeled Frame Exploitationcited by 27
  27. 2024Towards Open-Vocabulary Audio-Visual Event Localizationcited by 26
  28. 2023Cooperation Does Matter: Exploring Multi-Order Bilateral Relations for Audio-Visual Segmentationcited by 25
  29. 2021Self-supervised Audiovisual Representation Learning for Remote Sensing Datacited by 24
  30. 2022Static and Dynamic Concepts for Self-supervised Video Representation Learningcited by 23
  31. 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
  32. 2023Class-Incremental Grouping Network for Continual Audio-Visual Learningcited by 22
  33. 2023A Unified Audio-Visual Learning Framework for Localization, Separation, and Recognitioncited by 21
  34. 2024Stepping Stones: A Progressive Training Strategy for Audio-Visual Semantic Segmentationcited by 21
  35. 2023Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipationcited by 20
  36. 2024Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labelingcited by 20
  37. 2022Visual Sound Localization in the Wild by Cross-Modal Interference Erasingcited by 19
  38. 2023Hyperbolic Audio-visual Zero-shot Learningcited by 19

50 most cited of 118 All 118 (JSON)