Data to 7 Oct 2026

Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement

arXiv 2405.15973 ↗First posted 24 May 2024cs.CV cs.AI cs.CLcited by 74

On the map: LLM reasoning → Multimodal reasoning

also in 3 more

Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
Synthetic data and self-improvement for LLMs → Self-improving MLLMs and synthetic data

Abstract (arXiv)

Large vision-language models (LVLMs) have achieved impressive results in visual question-answering and reasoning tasks through vision instruction tuning on specific datasets. However, there remains significant room for improvement in aligning visual and language modalities. Existing methods often depend on external models or data, leading to uncontrollable and unstable alignment results. In this paper, we propose SIMA, a self-improvement framework that enhances visual and language modality alignment without external dependencies. SIMA leverages existing vision instruction tuning datasets to self-generate responses, incorporating an in-context self-critic mechanism that constructs preference pairs for tuning. Crucially, our approach allows LVLMs to act as critics by designing effective critic prompts, eliminating the need for additional fine-tuning with external instruction data. We introduce three novel visual metrics within the self-critic process to guide judgment, significantly improving the accuracy of self-critic. Through extensive experiments across 14 hallucination and comprehensive benchmarks, we demonstrate that SIMA significantly improves LVLM's performance and outperforms previous approaches, achieving superior modality alignment.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 52 of its 69 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

52 of 69 references.

Closest ancestors · 4

Same direction, newest first

Other references · 48

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2020Language Models are Few-Shot Learnerscited by 32,199
2014Microsoft COCO: Common Objects in Contextcited by 17,498
2022Training language models to follow instructions with human feedbackcited by 15,712
2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
2023Visual Instruction Tuningcited by 7,792
2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
2023Improved Baselines with Visual Instruction Tuningcited by 4,362

36 more

2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
2022Scaling Instruction-Finetuned Language Modelscited by 2,997
2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
2019Towards VQA Models That Can Readcited by 1,696
2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
2023VILA: On Pre-training for Visual Language Modelscited by 686
2018Object Hallucination in Image Captioningcited by 661
2024Self-Rewarding Language Modelscited by 654
2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
2023Silkie: Preference Distillation for Large Visual Language Modelscited by 128
2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedbackcited by 80
2023InstructZero: Efficient Instruction Optimization for Black-Box Large Language Modelscited by 68
2023How Many Demonstrations Do You Need for In-context Learning?cited by 52
2023Explore Spurious Correlations at the Concept Level in Language Models for Text Classificationcited by 48
2023When do you need Chain-of-Thought Prompting for ChatGPT?cited by 34
2024FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedbackcited by 30
2024Automated Data Curation for Robust Language Model Fine-Tuningcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 45; 2026: 10.

74 papers.

Sort:

By area:

2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
2024LLaVA-Critic: Learning to Evaluate Multimodal Modelscited by 132
2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 64
2024Self-Exploring Language Models: Active Preference Elicitation for Online Alignmentcited by 63
2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
2024Probing Visual Language Priors in VLMscited by 44
2024CREAM: Consistency Regularized Self-Rewarding Language Modelscited by 35
2025LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Modelcited by 34

38 more

2025Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimizationcited by 29
2024Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learningcited by 27
2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMscited by 25
2024Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehensioncited by 23
2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 23
2024LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentationcited by 22
2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizationcited by 21
2025Towards Visual Text Grounding of Multimodal Large Language Modelcited by 18
2025DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learningcited by 16
2025Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Followingcited by 14
2024VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoningcited by 13
2025MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEscited by 13
2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluationcited by 13
2024Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarioscited by 12
2025Aligning Multimodal LLM with Human Preference: A Surveycited by 12
2024Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolutioncited by 11
2025Anyprefer: An Agentic Framework for Preference Data Synthesiscited by 10
2025Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluationcited by 10
2024EACO: Enhancing Alignment in Multimodal LLMs via Critical Observationcited by 8
2025AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimizationcited by 8
2025Modality-Balancing Preference Optimization of Large Multimodal Models by Adversarial Negative Miningcited by 8
2025GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them?cited by 8
2025MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understandingcited by 7
2025DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Datacited by 6
2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gapcited by 6
2025Self-Improvement in Multimodal Large Language Models: A Surveycited by 6
2026iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Modelscited by 6
2024Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillationcited by 5
2025Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Modelscited by 5
2025Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinationscited by 5
2026Visual Preference Optimization with Rubric Rewardscited by 5
2026Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Modelscited by 5
2024Continual SFT Matches Multimodal RLHF with Negative Supervisioncited by 4
2025CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoningcited by 4
2025Fair Foundation Models for Medical Image Analysis: Challenges and Perspectivescited by 3
2025Improving Alignment in LVLMs with Debiased Self-Judgmentcited by 3
2025VLHSA: Vision-Language Hierarchical Semantic Alignment for Jigsaw Puzzle Solving with Eroded Gapscited by 3
2024Improving Visual Storytelling with Multimodal Large Language Modelscited by 2

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way