mlsanityData to 14 Oct 2026

OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference

arXiv 2502.18411 ↗First posted 25 Feb 2025cs.CVcited by 28

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

Recent advancements in open-source multi-modal large language models (MLLMs) have primarily focused on enhancing foundational capabilities, leaving a significant gap in human preference alignment. This paper introduces OmniAlign-V, a comprehensive dataset of 200K high-quality training samples featuring diverse images, complex questions, and varied response formats to improve MLLMs' alignment with human preferences. We also present MM-AlignBench, a human-annotated benchmark specifically designed to evaluate MLLMs' alignment with human values. Experimental results show that finetuning MLLMs with OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO), significantly enhances human preference alignment while maintaining or enhancing performance on standard VQA benchmarks, preserving their fundamental capabilities. Our datasets, benchmark, code and checkpoints have been released at https://github.com/PhoenixZ810/OmniAlign-V.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 69 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

37 of 69 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization15 Nov
  2. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data24 Oct
  3. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training10 Oct
  4. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  5. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs1 Jul
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
  8. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model29 Jan
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  10. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  11. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  12. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2023Segment Anythingcited by 8,234
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  12. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
13 more
  1. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  2. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  3. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  4. 2016A Diagram Is Worth A Dozen Imagescited by 804
  5. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  6. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  8. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  9. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  10. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  11. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferencescited by 76
  12. 2024An Open and Comprehensive Pipeline for Unified Object Grounding and Detectioncited by 65
  13. 2025Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinementcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 12.

28 papers.

Sort:

By area:

  1. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 139
  2. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  3. 2025MM-IFEngine: Towards Multimodal Instruction Followingcited by 42
  4. 2025Curriculum-RLAIF: Curriculum Alignment with Reinforcement Learning from AI Feedbackcited by 32
  5. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  6. 2025Advancing Complex Video Object Segmentation via Progressive Concept Constructioncited by 26
  7. 2025ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasingcited by 15
  8. 2025BaseReward: A Strong Baseline for Multimodal Reward Modelcited by 15
  9. 2025Aligning Multimodal LLM with Human Preference: A Surveycited by 13
  10. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11
  11. 2026ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratchcited by 10
  12. 2025Self-Improvement in Multimodal Large Language Models: A Surveycited by 6
16 more
  1. 2026MoD-DPO: Towards Mitigating Cross-modal Hallucinations in Omni LLMs using Modality Decoupled Preference Optimizationcited by 6
  2. 2026Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsingcited by 5
  3. 2025Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Modelscited by 4
  4. 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
  5. 2025GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devicescited by 3
  6. 2026Building a Precise Video Language with Human-AI Oversightcited by 3
  7. 2025Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Trainingcited by 2
  8. 2026The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotationcited by 2
  9. 2025Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Modelscited by 1
  10. 2025CoT4Det: A Chain-of-Thought Framework for Perception-Oriented Vision-Language Taskscited by 1
  11. 2026ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMscited by 1
  12. 2026Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understandingcited by 1
  13. 2026Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Gamescited by 0
  14. 2026The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrailscited by 0
  15. 2026Text-Anchored Semantic Perturbations for Transferable Jailbreak Attacks on Multimodal Large Language Modelscited by 0
  16. 2026Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Understanding Alignment in Multimodal LLMs: A Comprehensive Study
  2. 2024Self-Supervised Visual Preference Alignment
  3. 2026Preference Data Selection for Mitigating the Alignment Tax in Large Language Models
  4. 2024EACO: Enhancing Alignment in Multimodal LLMs via Critical Observation
  5. 2024A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques
  6. 2024Synth-Align: Improving Trustworthiness in Vision-Language Model with Synthetic Preference Data Alignment
  7. 2025MM-RLHF: The Next Step Forward in Multimodal LLM Alignment
  8. 2024MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time