mlsanityData to 14 Oct 2026

Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?

arXiv 2605.22109 ↗First posted 21 May 2026cs.AI cs.CV cs.CYcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) are increasingly deployed in human-facing roles where personality perception is critical, yet existing benchmarks evaluate this capability solely on numerical Big Five score prediction, leaving open whether models truly perceive personality through behavioral understanding or merely prejudge through superficial pattern matching. We address this gap with three contributions. (i) A new task: we formalize Grounded Personality Reasoning (GPR), which requires MLLMs to anchor each Big Five rating in observable evidence through a chain of rating, reasoning, and grounding. (ii) A new dataset: we release MM-OCEAN (1,104 videos, 5,320 MCQs), produced by a multi-agent pipeline with human verification, with timestamped behavioral observations, evidence-grounded trait analyses, and seven categories of cue-grounding MCQs. (iii) Benchmark and analysis: we design a three-tier evaluation (rating, reasoning, grounding) plus four sample-level failure-mode metrics: Prejudice Rate (PR), Confabulation Rate (CR), Integration-failure Rate (IR), and Holistic-grounding Rate (HR), and benchmark 27 MLLMs (13 closed, 14 open). The analysis uncovers a striking Prejudice Gap: across the field, 51% of correct ratings are not grounded in retrieved cues, and the Holistic-Grounding Rate spans only 0-33.5%. These findings expose a disconnect between getting the right score and reasoning for the right reason, charting a roadmap for grounded social cognition in MLLMs.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 73 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

42 of 73 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Xiaomi MiMo-VL-Miloco Technical Report19 Dec
  2. 2025Qwen3-VL Technical Report26 Nov
  3. 2025Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions20 Nov
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone3 Aug
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  7. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 35

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  8. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  9. 2018Datasheets for Datasetscited by 1,386
  10. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  11. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  12. 2016Modeling Context in Referring Expressionscited by 1,063
23 more
  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  2. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  3. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  4. 2025Qwen3-Omni Technical Reportcited by 497
  5. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  6. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  7. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 314
  8. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
  9. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
  10. 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 118
  11. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 63
  12. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  13. 2024EmoBench: Evaluating the Emotional Intelligence of Large Language Modelscited by 53
  14. 2021Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptationcited by 42
  15. 2026STEP3-VL-10B Technical Reportcited by 32
  16. 2020Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Datasetcited by 31
  17. 2024Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimationcited by 19
  18. 2015Automated Analysis and Prediction of Job Interview Performancecited by 15
  19. 2016Deep Impression: Audiovisual Deep Residual Networks for Multimodal Apparent Personality Trait Recognitioncited by 15
  20. 2025Towards Interactive Intelligence for Digital Humanscited by 10
  21. 2025Multi-speaker Attention Alignment for Multimodal Social Interactioncited by 9
  22. 2026SocialDirector: Training-Free Social Interaction Control for Multi-Person Video Generationcited by 4
  23. 2025Robot Character Generation and Adaptive Human-Robot Interaction with Personality Shapingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data
  2. 2025Evaluating LLM Alignment on Personality Inference from Real-World Interview Data
  3. 2025From Five Dimensions to Many: Large Language Models as Precise and Interpretable Psychological Profilers
  4. 2025Beyond Self-Reports: Multi-Observer Agents for Personality Assessment in Large Language Models
  5. 2025Can LLMs Infer Personality from Real World Conversations?
  6. 2025Beyond BFI: The CSI for Enhanced Reliability and Validity in Evaluating LLM Personality Traits
  7. 2026Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models
  8. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits