Multimodal Evaluator Preference Collapse: Cross-Modal Coupling in Self-Evolving Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
also in 1 more
Abstract (arXiv)
When AI agents use language models to evaluate their own outputs in a feedback loop, systematic biases emerge. We show that Evaluator Preference Collapse (EPC) is dramatically amplified in multimodal settings. Using GPT-4o to evaluate DeepSeek-chat across text and visual tasks, we find that a single strategy (step_by_step) absorbs 48.4% of all weight -- 3.2x the collapse observed in text-only self-evaluation -- while three visual-domain strategies receive only 9.1% combined weight. We then demonstrate a novel phenomenon we term cross-modal coupling: evaluator preferences acquired on one modality transfer to and corrupt strategy selection on another. Through a four-phase isolation training paradigm, we measure coupling coefficients and document strategy inversion -- the optimal strategy for a modality reverses after cross-modal exposure. A Phase 3 statistical validation across five evaluator configurations (N=80 total independent repetitions, ~35,000 API calls) with both text-proxy and real-image visual tasks finds: cross-model evaluation produces strong coupling (JSD~0.19-0.34), real-image inputs yield the most directionally consistent signal (mean gamma_{T->V}=1.145, gamma_{V->T}=0.937, 70% T->V, Cohen's d=0.56), and self-evaluation provides near-complete immunity -- 97% of runs (N=30) yield zero coupling (JSD=0.003, d=0.07). Three methodological ablations and multi-executor validation confirm the effect is not a structural artifact. We introduce the coupling matrix indexed by evaluator identity, release the MM-EPC framework, and identify cross-model evaluator architecture as the primary risk factor for preference drift. Code and data: https://github.com/aidless/mm-epc.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 33 references that are in the map. First posted on arXiv 15 Jun 2026.
Builds on
18 of 33 references.
References · 18
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2024GPT-4o System Cardcited by 4,467
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
6 more
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
- 2024Self-Rewarding Language Modelscited by 654
- 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
- 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
- 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Seeing is Believing? Evaluating Vision-Language Model Susceptibility in Agent-to-Agent Multimodal Persuasion
- 2026Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?
- 2026C3PO: Evaluating Cross-Modal Composition and Counterfactual Performance in Omnimodal Models
- 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey
- 2025InterMT: Multi-Turn Interleaved Preference Alignment with Human Feedback
- 2026A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents
- 2025VisiPruner: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs
- 2026PERSONAJUDGE: Simulating Individual Human Preference Judgments with Evaluator-Specific Demonstration Data