mlsanityData to 7 Oct 2026

VGGSound: A Large-scale Audio-Visual Dataset

arXiv 2004.14368 ↗First posted 29 Apr 2020cs.CV cs.SD eess.AScited by 641

Not on the map. Why?

Cited by 53 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Our goal is to collect a large-scale audio-visual dataset with low label noise from videos in the wild using computer vision techniques. The resulting dataset can be used for training and evaluating audio recognition models. We make three contributions. First, we propose a scalable pipeline based on computer vision techniques to create an audio dataset from open-source media. Our pipeline involves obtaining videos from YouTube; using image classification algorithms to localize audio-visual correspondence; and filtering out ambient noise using audio verification. Second, we use this pipeline to curate the VGGSound dataset consisting of more than 210k videos for 310 audio classes. Third, we investigate various Convolutional Neural Network~(CNN) architectures and aggregation approaches to establish audio recognition baselines for our new dataset. Compared to existing audio datasets, VGGSound ensures audio-visual correspondence and is collected under unconstrained conditions. Code and the dataset are available at http://www.robots.ox.ac.uk/~vgg/data/vggsound/

Key facts. Cited by 641 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 35; 2022: 58; 2023: 98; 2024: 130; 2025: 183; 2026: 137.

641 papers.

Sort:

By area:

  1. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  2. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  3. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  4. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  5. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  6. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  7. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
  8. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  9. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  10. 2022AudioGen: Textually Guided Audio Generationcited by 292
  11. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  12. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
38 more
  1. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  2. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  3. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 256
  4. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  5. 2025Kimi-Audio Technical Reportcited by 210
  6. 2021Localizing Visual Sounds the Hard Waycited by 169
  7. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  8. 2022Audio-Visual Segmentationcited by 148
  9. 2022Contrastive Audio-Visual Masked Autoencodercited by 142
  10. 2021Taming Visually Guided Sound Generationcited by 136
  11. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  12. 2023Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Modelscited by 133
  13. 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 117
  14. 2024Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Alignerscited by 111
  15. 2024FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Soundscited by 109
  16. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  17. 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 101
  18. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 101
  19. 2023Joint Audio and Speech Understandingcited by 99
  20. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
  21. 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 94
  22. 2025Controllable Video Generation: A Surveycited by 93
  23. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  24. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 84
  25. 2022Localizing Visual Sounds the Easy Waycited by 81
  26. 2022Learning Audio-Video Modalities from Image Captionscited by 80
  27. 2021Everything at Once -- Multi-modal Fusion Transformer for Video Retrievalcited by 79
  28. 2022HEAR: Holistic Evaluation of Audio Representationscited by 77
  29. 2023Audio-Visual LLM for Video Understandingcited by 76
  30. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 76
  31. 2023Audio-Visual Segmentation with Semanticscited by 73
  32. 2022A Closer Look at Weakly-Supervised Audio-Visual Source Localizationcited by 72
  33. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72
  34. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 72
  35. 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 71
  36. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  37. 2023Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptationcited by 68
  38. 2023V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Modelscited by 65

50 most cited of 641 All 641 (JSON)