mlsanityData to 7 Oct 2026

LAION-5B: An open large-scale dataset for training next generation image-text models

arXiv 2210.08402 ↗First posted 16 Oct 2022cs.CV cs.AI cs.LGcited by 4,131

Not on the map. Why?

Cited by 1,403 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Groundbreaking language-vision architectures like CLIP and DALL-E proved the utility of training on large amounts of noisy image-text data, without relying on expensive accurate labels used in standard vision unimodal supervised learning. The resulting models showed capabilities of strong text-guided image generation and transfer to downstream tasks, while performing remarkably at zero-shot classification with noteworthy out-of-distribution robustness. Since then, large-scale language-vision models like ALIGN, BASIC, GLIDE, Flamingo and Imagen made further improvements. Studying the training and capabilities of such models requires datasets containing billions of image-text pairs. Until now, no datasets of this size have been made openly available for the broader research community. To address this problem and democratize research on large-scale multi-modal models, we present LAION-5B - a dataset consisting of 5.85 billion CLIP-filtered image-text pairs, of which 2.32B contain English language. We show successful replication and fine-tuning of foundational models like CLIP, GLIDE and Stable Diffusion using the dataset, and discuss further experiments enabled with an openly available dataset of this scale. Additionally we provide several nearest neighbor indices, an improved web-interface for dataset exploration and subset generation, and detection scores for watermark, NSFW, and toxic content detection. Announcement page https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/

Key facts. Cited by 4,131 papers in the map (data through 7 Oct 2026). Builds on 64 of its 109 references that are in the map. First posted on arXiv 16 Oct 2022.

Builds on

64 of 109 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  8. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  13. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  14. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  15. 2018Model Cards for Model Reportingcited by 1,068
  16. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  17. 2021Scaling Vision Transformerscited by 993
  18. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  19. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  20. 2021Robust fine-tuning of zero-shot modelscited by 785
  21. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  22. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  23. 2016Generative Adversarial Text to Image Synthesiscited by 735
  24. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  25. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  26. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  27. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  28. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  29. 2018Do Better ImageNet Models Transfer Better?cited by 560
  30. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  31. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  32. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  33. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  34. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  35. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
  36. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  37. 2019Connecting Vision and Language with Localized Narrativescited by 225
  38. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 53; 2023: 911; 2024: 1408; 2025: 1155; 2026: 604.

4,131 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  8. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
38 more
  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  4. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  7. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  11. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  12. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  13. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  14. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  15. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  16. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  17. 2023A Survey on Multimodal Large Language Modelscited by 898
  18. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  19. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  20. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  21. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  22. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  23. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  24. 2023Adversarial Diffusion Distillationcited by 741
  25. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  26. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  27. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  28. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  29. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  30. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  31. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  32. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  33. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  34. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  35. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  36. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  37. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  38. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545

50 most cited of 4,131 All 4,131 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models
  2. 2025MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
  3. 2025BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries
  4. 2023Joint Adaptive Representations for Image-Language Learning
  5. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  6. 2024Unified Generative and Discriminative Training for Multi-modal Large Language Models
  7. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  8. 2025Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation