# Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks

arXiv: https://arxiv.org/abs/2410.04055 · HTML: https://mlsanity.com/paper/2410.04055/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 57 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 15 of 15)

- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [A Diagram Is Worth A Dozen Images](https://mlsanity.com/paper/1603.07396/) (2016)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://mlsanity.com/paper/2401.16420/) (2024)
- [Silkie: Preference Distillation for Large Visual Language Models](https://mlsanity.com/paper/2312.10665/) (2023)
- [Aligning LLMs with Individual Preferences via Interaction](https://mlsanity.com/paper/2410.03642/) (2024)
- [From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models](https://mlsanity.com/paper/2403.12027/) (2024)
- [Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement](https://mlsanity.com/paper/2402.11060/) (2024)
- [Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models](https://mlsanity.com/paper/2402.12563/) (2024)
- [Large Language Models have Intrinsic Self-Correction Ability](https://mlsanity.com/paper/2406.15673/) (2024)
- [Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT](https://mlsanity.com/paper/2409.01207/) (2024)
- [Identifying built environment factors influencing driver yielding behavior at unsignalized intersections: A naturalistic open-source dataset collected in Minnesota](https://mlsanity.com/paper/2312.15113/) (2023)

## Built on by (most cited first, 30 of 32)

- [Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning](https://mlsanity.com/paper/2411.18203/) (2024)
- [Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning](https://mlsanity.com/paper/2507.05255/) (2025)
- [VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues](https://mlsanity.com/paper/2502.12084/) (2025)
- [From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2509.25373/) (2025)
- [HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation](https://mlsanity.com/paper/2502.12148/) (2025)
- [SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning](https://mlsanity.com/paper/2508.06125/) (2025)
- [ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection](https://mlsanity.com/paper/2505.16475/) (2025)
- [MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration](https://mlsanity.com/paper/2505.23224/) (2025)
- [Reasoning Path Divergence: A New Metric and Curation Strategy to Unlock LLM Diverse Thinking](https://mlsanity.com/paper/2510.26122/) (2025)
- [Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward](https://mlsanity.com/paper/2504.16727/) (2025)
- [Self-Improvement in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2510.02665/) (2025)
- [LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs](https://mlsanity.com/paper/2507.16809/) (2025)
- [Interactive Learning for LLM Reasoning](https://mlsanity.com/paper/2509.26306/) (2025)
- [Self-Improving World Modelling with Latent Actions](https://mlsanity.com/paper/2602.06130/) (2026)
- [Skill-MAS: Evolving Meta-Skill for Automatic Multi-Agent Systems](https://mlsanity.com/paper/2606.18837/) (2026)
- [Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models](https://mlsanity.com/paper/2601.20687/) (2026)
- [Unified-MAS: Universally Generating Domain-Specific Nodes for Empowering Automatic Multi-Agent Systems](https://mlsanity.com/paper/2603.21475/) (2026)
- [CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction](https://mlsanity.com/paper/2601.17420/) (2026)
- [Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answering](https://mlsanity.com/paper/2508.11975/) (2025)
- [Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration](https://mlsanity.com/paper/2509.13919/) (2025)
- [Contemplative Artificial Intelligence](https://mlsanity.com/paper/2504.15125/) (2025)
- [Forward-Only Continual Learning](https://mlsanity.com/paper/2509.01533/) (2025)
- [Evaluating Self-Correcting Vision Agents Through Quantitative and Qualitative Metrics](https://mlsanity.com/paper/2601.11637/) (2026)
- [ReflectCAP: Detailed Image Captioning with Reflective Memory](https://mlsanity.com/paper/2604.12357/) (2026)
- [Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning](https://mlsanity.com/paper/2606.29425/) (2026)
- [Through the LLM Looking Glass: A Socratic Probing of Donkeys, Elephants, and Markets](https://mlsanity.com/paper/2503.16674/) (2025)
- [A Rolling Stone Gathers No Moss: Adaptive Policy Optimization for Stable Self-Evaluation in Large Multimodal Models](https://mlsanity.com/paper/2508.09155/) (2025)
- [Recompute or Reuse? Diagnosing and Mitigating Textual Shortcuts in VLM Self-Reflection](https://mlsanity.com/paper/2608.01930/) (2026)
- [Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding](https://mlsanity.com/paper/2609.04741/) (2026)
- [Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Ability](https://mlsanity.com/paper/2609.24760/) (2026)

## Abstract (arXiv)

While Vision-Language Models (VLMs) have shown remarkable abilities in visual and language reasoning tasks, they invariably generate flawed responses. Self-correction that instructs models to refine their outputs presents a promising solution to this issue. Previous studies have mainly concentrated on Large Language Models (LLMs), while the self-correction abilities of VLMs, particularly concerning both visual and linguistic information, remain largely unexamined. This study investigates the self-correction capabilities of VLMs during both inference and fine-tuning stages. We introduce a Self-Correction Learning (SCL) approach that enables VLMs to learn from their self-generated self-correction data through Direct Preference Optimization (DPO) without relying on external feedback, facilitating self-improvement. Specifically, we collect preferred and disfavored samples based on the correctness of initial and refined responses, which are obtained by two-turn self-correction with VLMs during the inference stage. Experimental results demonstrate that although VLMs struggle to self-correct effectively during iterative inference without additional fine-tuning and external feedback, they can enhance their performance and avoid previous mistakes through preference fine-tuning when their self-generated self-correction data are categorized into preferred and disfavored samples. This study emphasizes that self-correction is not merely a refinement process; rather, it should enhance the reasoning abilities of models through additional training, enabling them to generate high-quality responses directly without further refinement.