mlsanityData to 7 Oct 2026

Vision-Language Models Can Self-Improve Reasoning via Reflection

arXiv 2411.00855 ↗First posted 30 Oct 2024cs.LG cs.AI cs.CLcited by 66

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Chain-of-thought (CoT) has proven to improve the reasoning capability of large language models (LLMs). However, due to the complexity of multimodal scenarios and the difficulty in collecting high-quality CoT data, CoT reasoning in multimodal LLMs has been largely overlooked. To this end, we propose a simple yet effective self-training framework, R3V, which iteratively enhances the model's Vision-language Reasoning by Reflecting on CoT Rationales. Our framework consists of two interleaved parts: (1) iteratively bootstrapping positive and negative solutions for reasoning datasets, and (2) reflection on rationale for learning from mistakes. Specifically, we introduce the self-refine and self-select losses, enabling the model to refine flawed rationale and derive the correct answer by comparing rationale candidates. Experiments on a wide range of vision-language tasks show that R3V consistently improves multimodal LLM reasoning, achieving a relative improvement of 23 to 60 percent over GPT-distilled baselines. Additionally, our approach supports self-reflection on generated solutions, further boosting performance through test-time computation.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 55 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

40 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Training22 Apr
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  3. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning10 Aug
  4. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  12. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
24 more
  1. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  2. 2022Large Language Models Can Self-Improvecited by 656
  3. 2024Self-Rewarding Language Modelscited by 654
  4. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  5. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  6. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  7. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  8. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  9. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  10. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  11. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
  12. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 214
  13. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  14. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  15. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  16. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  17. 2024Provably Robust DPO: Aligning Language Models with Noisy Feedbackcited by 133
  18. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  19. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  20. 2024Enhancing Large Vision Language Models with Self-Training on Image Comprehensioncited by 70
  21. 2024EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understandingcited by 32
  22. 2024Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoningcited by 26
  23. 2024Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervisioncited by 24
  24. 2023Beyond Generic: Enhancing Image Captioning with Real-World Knowledge using Vision-Language Pre-Training Modelcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 41; 2026: 21.

66 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
  3. 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 89
  4. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  5. 2025Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulationcited by 40
  6. 2025CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Eracited by 40
  7. 2025First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Trainingcited by 34
  8. 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 31
  9. 2025Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoningcited by 24
  10. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  11. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  12. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
38 more
  1. 2025OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflowscited by 18
  2. 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Modelscited by 16
  3. 2025Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generationcited by 14
  4. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  5. 2025Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Riskcited by 13
  6. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 12
  7. 2025A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Modelscited by 12
  8. 2025ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflectioncited by 11
  9. 2025Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questionscited by 11
  10. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimizationcited by 10
  11. 2025Comparative Analysis Based on DeepSeek, ChatGPT, and Google Gemini: Features, Techniques, Performance, Future Prospectscited by 9
  12. 2025Can GRPO Boost Complex Multimodal Table Understanding?cited by 9
  13. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agentscited by 9
  14. 2025Toward Effective Reinforcement Learning Fine-Tuning for Medical VQA in Vision-Language Modelscited by 8
  15. 2025Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognitioncited by 7
  16. 2025VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanismcited by 7
  17. 2026Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflictscited by 7
  18. 2026TIDE: Trajectory-based Diagnostic Evaluation of Test-Time Improvement in LLM Agentscited by 7
  19. 2025From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignmentcited by 6
  20. 2025OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?cited by 6
  21. 2025Self-Improvement in Multimodal Large Language Models: A Surveycited by 6
  22. 2026iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Modelscited by 6
  23. 2026PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignmentcited by 5
  24. 2025RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferencescited by 4
  25. 2025Making VLMs More Robot-Friendly: Self-Critical Distillation of Low-Level Procedural Reasoningcited by 4
  26. 2025A Survey of Deep Learning for Geometry Problem Solvingcited by 4
  27. 2026Seeing Farther and Smarter: Value-Guided Multi-Path Reflection for VLM Policy Optimizationcited by 4
  28. 2026Vision Language Models Cannot Reason About Physical Transformationcited by 4
  29. 2025Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neurosciencecited by 3
  30. 2026Human Simulation Computation: A Human-Inspired Framework for Adaptive AI Systemscited by 3
  31. 2026Egocentric Bias in Vision-Language Modelscited by 3
  32. 2024VisGraphVar: A Benchmark Generator for Assessing Variability in Graph Analysis Using Large Vision-Language Modelscited by 2
  33. 2025InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoningcited by 2
  34. 2025Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibrationcited by 2
  35. 2025DF-LLaVA: Unlocking MLLMs for Synthetic Image Detection via Knowledge Injection and Conflict-Driven Self-Reflectioncited by 2
  36. 2026Speech-Hands: A Self-Reflection Voice Agentic Approach to Speech Recognition and Audio Reasoning with Omni Perceptioncited by 2
  37. 2024EVLM: Self-Reflective Multimodal Reasoning for Cross-Dimensional Visual Editingcited by 1
  38. 2025Med-R2: Perception and Reflection-driven Complex Reasoning for Medical Report Generationcited by 1

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025See, Think, Learn: A Self-Taught Multimodal Reasoner
  2. 2026Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
  3. 2025Sherlock: Self-Correcting Reasoning in Vision-Language Models
  4. 2025The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs
  5. 2025Corvid: Improving Multimodal Large Language Models Towards Chain-of-Thought Reasoning
  6. 2025SDRT: Enhance Vision-Language Models by Self-Distillation with Diverse Reasoning Traces
  7. 2025Coherent Multimodal Reasoning with Iterative Self-Evaluation for Vision-Language Models
  8. 2025Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought