mlsanityData to 14 Oct 2026

LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-training

arXiv 2608.24845 ↗First posted 25 Aug 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present LAION-BVD, a large-scale open video dataset for multimodal learning, which contains 1.3B platform-specific video URLs collected from CommonCrawl. From these, we download 80M videos with a total duration of 10 million hours. The dataset is designed for multimodal pre-training across the video, audio, and image modalities. Using content-aware scene detection, we extract clips for which we synthetically generate video and audio captions. Models trained on these data achieve competitive performance on standard video-text and audio-text benchmarks, with consistent improvements as training or model scale increases. Additionally, we explore video frames as an alternative source of image-text data by extracting scene-changing frames. These frames exhibit a visual distribution distinct from standard web image corpora, and models trained on this dataset achieve strong image-text retrieval performance. We release LAION-BVD to the research community. It significantly expands open access to multimodal videos at an unprecedented scale.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 63 of its 171 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

63 of 171 references.

References · 63

  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  10. 2017The Kinetics Human Action Video Datasetcited by 1,909
  11. 2017Decoupled Weight Decay Regularizationcited by 1,831
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
38 more
  1. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  2. 2021Grounded Language-Image Pre-trainingcited by 1,156
  3. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  4. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  5. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  6. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  7. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  8. 2017Localizing Moments in Video with Natural Languagecited by 663
  9. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  10. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  11. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  12. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  13. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  14. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  15. 2025Qwen3-Omni Technical Reportcited by 497
  16. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  17. 2017Non-Autoregressive Neural Machine Translationcited by 463
  18. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 334
  19. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  20. 2023Demystifying CLIP Datacited by 270
  21. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237
  22. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
  23. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
  24. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 172
  25. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 154
  26. 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 138
  27. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 128
  28. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 116
  29. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 87
  30. 2023Tag2Text: Guiding Vision-Language Model via Image Taggingcited by 86
  31. 2022VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captionerscited by 70
  32. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 68
  33. 2023ChatBridge: Bridging Modalities with Large Language Model as a Language Catalystcited by 66
  34. 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 62
  35. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 59
  36. 2025Scalable Vision Language Model Training via High Quality Data Curationcited by 55
  37. 2024LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videoscited by 51
  38. 2023WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Modelscited by 50

50 most cited of 63 All 63 (JSON)

Built on by

Not cited yet.