mlsanityData to 14 Oct 2026

MixGen: A New Multi-Modal Data Augmentation

arXiv 2206.08358 ↗First posted 16 Jun 2022cs.CV cs.AI cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

Data augmentation is a necessity to enhance data efficiency in deep learning. For vision-language pre-training, data is only augmented either for images or for text in previous works. In this paper, we present MixGen: a joint data augmentation for vision-language representation learning to further improve data efficiency. It generates new image-text pairs with semantic relationships preserved by interpolating images and concatenating text. It's simple, and can be plug-and-played into existing pipelines. We evaluate MixGen on four architectures, including CLIP, ViLT, ALBEF and TCL, across five downstream vision-language tasks to show its versatility and effectiveness. For example, adding MixGen in ALBEF pre-training leads to absolute performance improvements on downstream tasks: image-text retrieval (+6.2% on COCO fine-tuned and +5.3% on Flicker30K zero-shot), visual grounding (+0.9% on RefCOCO+), visual reasoning (+0.9% on NLVR2), visual question answering (+0.3% on VQA2.0), and visual entailment (+0.4% on SNLI-VE).

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Builds on 56 of its 72 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

56 of 72 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  10. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
38 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  2. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  3. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  4. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  5. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  6. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  7. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  9. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  10. 2017Random Erasing Data Augmentationcited by 1,239
  11. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  13. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  14. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  15. 2017Mixed Precision Trainingcited by 1,156
  16. 2021Grounded Language-Image Pre-trainingcited by 1,156
  17. 2016Modeling Context in Referring Expressionscited by 1,063
  18. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  19. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  20. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  21. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  22. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  23. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  24. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  25. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  26. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  27. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 459
  28. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  29. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  30. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  31. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 261
  32. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  33. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 252
  34. 2020UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learningcited by 239
  35. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  36. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  37. 2020SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularizationcited by 149
  38. 2020ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Datacited by 137

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 18; 2024: 15; 2025: 16; 2026: 4.

61 papers.

Sort:

By area:

  1. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
  2. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Conceptscited by 101
  3. 2022Teaching Structured Vision&Language Concepts to Vision&Language Modelscited by 71
  4. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  5. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 41
  6. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  7. 2023Measuring Progress in Fine-grained Vision-and-Language Understandingcited by 25
  8. 2022Unifying Vision-Language Representation Space with Single-tower Transformercited by 22
  9. 2024A Survey on Mixup Augmentations and Beyondcited by 20
  10. 2022Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignmentcited by 19
  11. 2022Learning Multimodal Data Augmentation in Feature Spacecited by 18
  12. 2024Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classificationcited by 18
38 more
  1. 2022YORO -- Lightweight End to End Visual Groundingcited by 17
  2. 2023Cross-modality Data Augmentation for End-to-End Sign Language Translationcited by 16
  3. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  4. 2024Enhancing Vision-Language Pre-training with Rich Supervisionscited by 13
  5. 2022Exploring Train and Test-Time Augmentations for Audio-Language Learningcited by 11
  6. 2023Data-Efficient Multimodal Fusion on a Single GPUcited by 11
  7. 2023MAP: A Model-agnostic Pretraining Framework for Click-through Rate Predictioncited by 10
  8. 2022Multimodal Data Augmentation for Visual-Infrared Person ReID with Corrupted Datacited by 9
  9. 2024A Survey of Data Synthesis Approachescited by 9
  10. 2025Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representationscited by 9
  11. 2023Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learningcited by 8
  12. 2024Enhancing Adversarial Robustness of Vision-Language Models through Low-Rank Adaptationcited by 8
  13. 2025Synergistic Prompting for Robust Visual Recognition with Missing Modalitiescited by 8
  14. 20233VL: Using Trees to Improve Vision-Language Models' Interpretabilitycited by 7
  15. 2024Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentationcited by 7
  16. 2023Improved Automatic Diabetic Retinopathy Severity Classification Using Deep Multimodal Fusion of UWF-CFP and OCTA Imagescited by 6
  17. 2024Does VLM Classification Benefit from LLM Description Semantics?cited by 6
  18. 2022Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learningcited by 4
  19. 2023Multiple-Question Multiple-Answer Text-VQAcited by 4
  20. 2025MIDAS: Mixing Ambiguous Data with Soft Labels for Dynamic Facial Expression Recognitioncited by 4
  21. 2025Uneven Event Modeling for Partially Relevant Video Retrievalcited by 4
  22. 2023A Multi-Modal Multilingual Benchmark for Document Image Classificationcited by 3
  23. 2024Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching abilitycited by 3
  24. 2024BCTR: Bidirectional Conditioning Transformer for Scene Graph Generationcited by 3
  25. 2024FTF-ER: Feature-Topology Fusion-Based Experience Replay Method for Continual Graph Learningcited by 3
  26. 2025Contrastive Visual Data Augmentationcited by 3
  27. 2025World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Modelscited by 3
  28. 2023A Guide for Practical Use of ADMG Causal Data Augmentationcited by 2
  29. 2023Selective Volume Mixup for Video Action Recognitioncited by 2
  30. 2024ARMADA: Attribute-Based Multimodal Data Augmentationcited by 2
  31. 2024Harnessing Shared Relations via Multimodal Mixup Contrastive Learning for Multimodal Classificationcited by 2
  32. 2025Hear-Your-Click: Interactive Object-Specific Video-to-Audio Generationcited by 2
  33. 2025MIDAS: Misalignment-based Data Augmentation Strategy for Imbalanced Multimodal Learningcited by 2
  34. 2023Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detectioncited by 1
  35. 2023PowMix: A Versatile Regularizer for Multimodal Sentiment Analysiscited by 1
  36. 2025SynRES: Towards Referring Expression Segmentation in the Wild via Synthetic Datacited by 1
  37. 2025Enhancing Ambiguous Dynamic Facial Expression Recognition with Soft Label-based Data Augmentationcited by 1
  38. 2025ReFineG: Synergizing Small Supervised Models and LLMs for Low-Resource Grounded Multimodal NERcited by 1

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Dataset Augmentation by Mixing Visual Concepts
  2. 2023TiMix: Text-aware Image Mixing for Effective Vision-Language Pre-training
  3. 2025MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding
  4. 2025Context-guided Responsible Data Augmentation with Diffusion Models
  5. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models
  6. 2023CiT: Curation in Training for Effective Vision-Language Data
  7. 2026MaD-Mix: Multi-Modal Data Mixtures via Latent Space Coupling for Vision-Language Model Training
  8. 2025VACoT: Rethinking Visual Data Augmentation with VLMs