mlsanityData to 14 Oct 2026

Data Selection Through Iterative Self-Filtering for Vision-Language Settings

arXiv 2606.23611 ↗First posted 22 Jun 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The availability of large amounts of clean data is paramount to training neural networks. However, at large scales, manual oversight is impractical, resulting in sizeable datasets that can be very noisy. Attempts to mitigate this obstacle to producing performant vision-language models have so far involved heuristics, curated reference datasets, and using pre-trained models. Here we propose a novel, bootstrapped method in which a CLIP model is trained on an evolving, self-selected dataset. This evolving dataset constitutes a balance of filtered, highly probable clean samples as well as diverse samples from the entire distribution. Our proposed Self-Filtering method iterates between training the model and selecting a subsequently improved data mixture. Training on vision-language datasets filtered by the proposed approach improves downstream performance without the need for additional data or pre-trained models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 53 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

43 of 53 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  8. 2013Describing Textures in the Wildcited by 2,125
  9. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  10. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
31 more
  1. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  2. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  3. 2017A Closer Look at Memorization in Deep Networkscited by 942
  4. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  5. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  6. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  7. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
  8. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  9. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  10. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  11. 2022Visual Classification via Description from Large Language Modelscited by 336
  12. 2023Data Selection for Language Models via Importance Resamplingcited by 329
  13. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  14. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 303
  15. 2023Demystifying CLIP Datacited by 270
  16. 2023Data Filtering Networkscited by 250
  17. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  18. 2024A Survey on Data Selection for Language Modelscited by 236
  19. 2017Decoupling "when to update" from "how to update"cited by 230
  20. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
  21. 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
  22. 2024Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnosticcited by 77
  23. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  24. 2023T-MARS: Improving Visual Representations by Circumventing Text Feature Learningcited by 43
  25. 2023TiC-CLIP: Continual Training of CLIP Modelscited by 38
  26. 2024Effective pruning of web-scale datasets based on complexity of concept clusterscited by 34
  27. 2024Data curation via joint example selection further accelerates multimodal learningcited by 33
  28. 2023Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understandingcited by 28
  29. 2024CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learningcited by 20
  30. 2023The Devil is in the Details: A Deep Dive into the Rabbit Hole of Data Filteringcited by 19
  31. 2024Collapse of Self-trained Language Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection
  2. 2022Localized Latent Updates for Fine-Tuning Vision-Language Models
  3. 2024BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping
  4. 2026Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning
  5. 2024Efficient Test-Time Prompt Tuning for Vision-Language Models
  6. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters
  7. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models
  8. 2024Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization