mlsanityData to 14 Oct 2026

The Effects of Visual Priming on Cooperative Behavior in Vision-Language Models

arXiv 2604.27953 ↗First posted 30 Apr 2026cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

As Vision-Language Models (VLMs) become increasingly integrated into decision-making systems, it is essential to understand how visual inputs influence their behavior. This paper investigates the effects of visual priming on VLMs' cooperative behavior using the Iterated Prisoner's Dilemma (IPD) as a test scenario. We examine whether exposure to images depicting behavioral concepts (kindness/helpfulness vs. aggressiveness/selfishness) and color-coded reward matrices alters VLM decision patterns. Experiments were conducted across multiple state-of-the-art VLMs. We further explore mitigation strategies including prompt modifications, Chain of Thought (CoT) reasoning, and visual token reduction. Results show that VLM behavior can be influenced by both image content and color cues, with varying susceptibility and mitigation effectiveness across models. These findings not only underscore the importance of robust evaluation frameworks for VLM deployment in visually rich and safety-critical environments, but also highlight how architectural and training differences among models may lead to distinct behavioral responses-an area worthy of further investigation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 34 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

19 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025On the robustness of multimodal language model towards distractions13 Feb
  2. 2024Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts24 Jun

Other references · 17

  1. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  2. 2020Quantifying Attention Flow in Transformerscited by 775
  3. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  4. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  5. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 301
  6. 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 63
  7. 2024Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometricscited by 59
  8. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 56
  9. 2023Epidemic Modeling with Generative Agentscited by 49
  10. 2024Do Language Models Exhibit Human-like Structural Priming Effects?cited by 21
  11. 2024AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessmentcited by 20
  12. 2023Towards the Scalable Evaluation of Cooperativeness in Language Modelscited by 12
5 more
  1. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reductioncited by 8
  2. 2024Prompt-Aware Adapter: Towards Learning Adaptive Visual Tokens for Multimodal Large Language Modelscited by 3
  3. 2025Intrinsic Model Weaknesses: How Priming Attacks Unveil Vulnerabilities in Large Language Modelscited by 3
  4. 2025Identifying Cooperative Personalities in Multi-agent Contexts through Personality Steering with Representation Engineeringcited by 2
  5. 2025When Less is Enough: Adaptive Token Reduction for Efficient Image Representationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Visual Persuasion: What Influences Decisions of Vision-Language Models?
  2. 2026Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMs
  3. 2025Test-Time Reasoning Through Visual Human Preferences with VLMs and Soft Rewards
  4. 2025VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments
  5. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict
  6. 2025Not There Yet: Evaluating Vision Language Models in Simulating the Visual Perception of People with Low Vision
  7. 2025Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects
  8. 2025Reinforcement Learning for Large Model: A Survey