mlsanityData to 14 Oct 2026

MM-MoralBench: A MultiModal Moral Evaluation Benchmark for Large Vision-Language Models

arXiv 2412.20718 ↗First posted 30 Dec 2024cs.CV cs.AIcited by 9

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The rapid integration of Large Vision-Language Models (LVLMs) into critical domains necessitates comprehensive moral evaluation to ensure their alignment with human values. While extensive research has addressed moral evaluation in LLMs, text-centric assessments cannot adequately capture the complex contextual nuances and ambiguities introduced by visual modalities. To bridge this gap, we introduce MM-MoralBench, a multimodal moral evaluation benchmark grounded in Moral Foundations Theory. We construct unique multimodal scenarios by combining synthesized visual contexts with character dialogues to simulate real-world dilemmas where visual and linguistic information interact dynamically. Our benchmark assesses models across six moral foundations through moral judgment, classification, and response tasks. Extensive evaluations of over 20 LVLMs reveal that models exhibit pronounced moral alignment bias, diverging significantly from human consensus. Furthermore, our analysis indicates that general scaling or structural improvements yield diminishing returns in moral alignment, and thinking paradigm may trigger overthinking-induced failures in moral contexts, highlighting the necessity for targeted moral alignment strategies. Our benchmark is publicly available.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 40 references that are in the map. First posted on arXiv 30 Dec 2024.

Builds on

28 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug
  2. 2023Vision-Language Models for Vision Tasks: A Survey3 Apr

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  7. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  8. 2020Aligning AI With Shared Human Valuescited by 723
  9. 2024Yi: Open Foundation Models by 01.AIcited by 719
  10. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  11. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  12. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
14 more
  1. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  2. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  3. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  4. 2023Large Language Model Alignment: A Surveycited by 243
  5. 2023Large Language Models in Finance: A Surveycited by 226
  6. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  7. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
  8. 2023Large Language Models in Law: A Surveycited by 128
  9. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 101
  10. 2022When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgmentcited by 93
  11. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 59
  12. 2024MoralBench: Moral Evaluation of LLMscited by 50
  13. 2024CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Modelscited by 10
  14. 2023Unpacking the Ethical Value Alignment in Big Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 3.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models
  2. 2026Do VLMs Have a Moral Backbone? A Study on the Fragile Morality of Vision-Language Models
  3. 2026Tracing Moral Foundations in Large Language Models
  4. 2026Position: Evaluations of AI Moral Reasoning Still Miss Half of the Picture
  5. 2026The Fragility Of Moral Judgment In Large Language Models
  6. 2026Unsupervised Elicitation of Moral Values from Language Models
  7. 2025EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models
  8. 2026Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments