mlsanityData to 14 Oct 2026

Can a Unimodal Language Agent Provide Preferences to Tune a Multimodal Vision-Language Model?

arXiv 2601.06424 ↗First posted 10 Jan 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 2 more
Abstract (arXiv)

To explore a more scalable path for adding multimodal capabilities to existing LLMs, this paper addresses a fundamental question: Can a unimodal LLM, relying solely on text, reason about its own informational needs and provide effective feedback to optimize a multimodal model? To answer this, we propose a method that enables a language agent to give feedback to a vision-language model (VLM) to adapt text generation to the agent's preferences. Our results from different experiments affirm this hypothesis, showing that LLM preference feedback significantly enhances VLM descriptions. Using our proposed method, we find that the VLM can generate multimodal scene descriptions to help the LLM better understand multimodal context, leading to improvements of maximum 13% in absolute accuracy compared to the baseline multimodal approach. Furthermore, a human study validated our AI-driven feedback, showing a 64.6% preference alignment rate between the LLM's choices and human judgments. Extensive experiments provide insights on how and why the method works and its limitations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 45 references that are in the map. First posted on arXiv 10 Jan 2026.

Builds on

20 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward1 Apr
  2. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2020Learning to summarize from human feedbackcited by 2,588
  8. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  9. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 301
  10. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  11. 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 174
  12. 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 132
6 more
  1. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  2. 2022Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancementcited by 27
  3. 2025Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMscited by 19
  4. 2024End User Authoring of Personalized Content Classifiers: Comparing Example Labeling, Rule Writing, and LLM Promptingcited by 8
  5. 2023TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Modelscited by 6
  6. 2025VC-LLM: Automated Advertisement Video Creation from Raw Footage using Multi-modal LLMscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  2. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  3. 2024Large Multimodal Agents: A Survey
  4. 2026Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
  5. 2025Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaboration
  6. 2025Transferring Textual Preferences to Vision-Language Understanding through Model Merging
  7. 2024Why do LLaVA Vision-Language Models Reply to Images in English?
  8. 2025Mordal: Automated Pretrained Model Selection for Vision Language Models