mlsanityData to 7 Oct 2026

VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values

arXiv 2407.03000 ↗First posted 3 Jul 2024cs.CL cs.CVcited by 12

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large vision language models (VLMs) have demonstrated significant potential for integration into daily life, making it crucial for them to incorporate human values when making decisions in real-world situations. This paper introduces VIVA, a benchmark for VIsion-grounded decision-making driven by human VAlues. While most large VLMs focus on physical-level skills, our work is the first to examine their multimodal capabilities in leveraging human values to make decisions under a vision-depicted situation. VIVA contains 1,240 images depicting diverse real-world situations and the manually annotated decisions grounded in them. Given an image there, the model should select the most appropriate action to address the situation and provide the relevant human values and reason underlying the decision. Extensive experiments based on VIVA show the limitation of VLMs in using human values to make multimodal decisions. Further analyses indicate the potential benefits of exploiting action consequences and predicted human values.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 58 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

31 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug
  3. 2023Vision-Language Models for Vision Tasks: A Survey3 Apr

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  10. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  11. 2016A Diagram Is Worth A Dozen Imagescited by 793
  12. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
16 more
  1. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  3. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  4. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  5. 2022Is GPT-3 a Good Data Annotator?cited by 208
  6. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 192
  7. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  8. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  9. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  10. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 92
  11. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
  12. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  13. 2024PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chaincited by 29
  14. 2023Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailmentcited by 22
  15. 2022MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussionscited by 17
  16. 2024Harnessing the power of LLMs for normative reasoning in MASscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 2.

12 papers.

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2025MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Modelscited by 10
  3. 2025SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigationcited by 10
  4. 2025Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learningcited by 9
  5. 2025OWLViz: An Open-World Benchmark for Visual Question Answeringcited by 3
  6. 2025From Evaluation to Defense: Advancing Safety in Video Large Language Modelscited by 2
  7. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  8. 2025ViMRHP: A Vietnamese Benchmark Dataset for Multimodal Review Helpfulness Prediction via Human-AI Collaborative Annotationcited by 0
  9. 2025MoralCLIP: Contrastive Alignment of Vision-and-Language Representations with Moral Foundations Theorycited by 0
  10. 2025VIVA+: Human-Centered Situational Decision-Makingcited by 0
  11. 2025Spot The Ball: A Benchmark for Visual Social Inferencecited by 0
  12. 2026MAVEN: A Macro-Societal Value Evaluation Framework of Multimodal Content with Compact Aligned Evaluatorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VisualActBench: Can VLMs See and Act like a Human?
  2. 2026Evaluating VLMs on Multimodal Aristotelian Persuasion Tasks
  3. 2026A Dataset for Dynamic Human Preferences for Vision Language Models
  4. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
  5. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  6. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences
  7. 2025Measuring How (Not Just Whether) VLMs Build Common Ground
  8. 2026VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation