mlsanityData to 7 Oct 2026

Self-Improvement in Multimodal Large Language Models: A Survey

arXiv 2510.02665 ↗First posted 3 Oct 2025cs.CLcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

Recent advancements in self-improvement for Large Language Models (LLMs) have efficiently enhanced model capabilities without significantly increasing costs, particularly in terms of human effort. While this area is still relatively young, its extension to the multimodal domain holds immense potential for leveraging diverse data sources and developing more general self-improving models. This survey is the first to provide a comprehensive overview of self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview of the current literature and discuss methods from three perspectives: 1) data collection, 2) data organization, and 3) model optimization, to facilitate the further development of self-improvement in MLLMs. We also include commonly used evaluations and downstream applications. Finally, we conclude by outlining open challenges and future research directions.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 84 of its 119 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

84 of 119 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner6 Mar
  2. 2025OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference25 Feb
  3. 2024Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution20 Dec
  4. 2024Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach26 Nov
  5. 2024MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct9 Sep
  6. 2024Understanding Alignment in Multimodal LLMs: A Comprehensive Study2 Jul
  7. 2024Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback6 Feb
  8. 2023Silkie: Preference Distillation for Large Visual Language Models17 Dec
  9. 2023Aligning Large Multimodal Models with Factually Augmented RLHF25 Sep
  10. 2023VIGC: Visual Instruction Generation and Correction24 Aug
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 73

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  8. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  9. 2025Qwen2.5-Omni Technical Reportcited by 704
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  11. 2018Object Hallucination in Image Captioningcited by 661
  12. 2022Large Language Models Can Self-Improvecited by 656
38 more
  1. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  2. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  3. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  4. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  5. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  6. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  7. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  8. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  9. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  10. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  11. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  12. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  13. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 293
  14. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
  15. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  16. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  17. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 196
  18. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 177
  19. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 163
  20. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  21. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  22. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  23. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Enginecited by 111
  24. 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 99
  25. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedbackcited by 80
  26. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvementcited by 74
  27. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
  28. 2024Enhancing Large Vision Language Models with Self-Training on Image Comprehensioncited by 70
  29. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learningcited by 68
  30. 2025VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learningcited by 68
  31. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
  32. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 66
  33. 2024Comprehensive Exploration of Synthetic Data Generation: A Surveycited by 59
  34. 2024VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignmentcited by 56
  35. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMscited by 49
  36. 2024Probing Visual Language Priors in VLMscited by 44
  37. 2024SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmarkcited by 35
  38. 2024Diving into Self-Evolving Training for Multimodal Reasoningcited by 31

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environmentscited by 4
  2. 2026Mitigating loss of control in advanced AI systems through instrumental goal trajectoriescited by 1
  3. 2026RISE: Reliable Improvement in Self-Evolving Vision-Language Modelscited by 1
  4. 2025An Aristotelian ontology of instrumental goals: Structural features to be managed and not failures to be eliminatedcited by 0
  5. 2026A three-dimensional typology of agency for advanced AI systemscited by 0
  6. 2026Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlook
  2. 2024MM-LLMs: Recent Advances in MultiModal Large Language Models
  3. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
  4. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models
  5. 2024Personalized Multimodal Large Language Models: A Survey
  6. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  7. 2023A Comprehensive Overview of Large Language Models
  8. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages