mlsanityData to 14 Oct 2026

MAVEN: A Macro-Societal Value Evaluation Framework of Multimodal Content with Compact Aligned Evaluators

arXiv 2608.18096 ↗First posted 8 Jun 2026cs.CL cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Assessing whether multimodal content aligns with macro-societal values, such as peace, justice, and freedom, has become an increasingly urgent challenge. Existing frameworks are largely confined to safety-oriented taxonomies, text-only psychometric probes, or single-label classification. Therefore, we propose MAVEN, a hierarchical framework for macro-societal value evaluation of multimodal content, grounded in international human-rights instruments and cultural value theory. MAVEN organizes values into 6 primary dimensions and 72 secondary indicators, supporting multi-level quantitative scoring. Building on MAVEN, we construct a human-verified multimodal benchmark and a soft-match metric to evaluate VLMs' assessments across value dimensions. For evaluator optimization, we propose a span-adaptive variant of multi-level preference optimization for evaluator distillation, together with a training-free multi-role consensus strategy at inference time. We evaluate existing open- and closed-source VLMs on our benchmark, revealing shared tendencies and clear differences in macro-societal value judgments. Experiments show that our compact 2B evaluator matches its 8B counterpart in the same family and approaches frontier closed-source VLMs, offering a practical path toward scalable macro-societal value evaluation. Our SA-MDPO implementation and MacroValue-Bench are available at https://github.com/zzzzzzzzjj/MAVEN.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 12 of its 36 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

12 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale6 Jan
  2. 2024ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs15 Sep
  3. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
  4. 2020Aligning AI With Shared Human Values5 Aug

Other references · 8

  1. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  2. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  3. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
  4. 2024SLM-Mod: Small Language Models Surpass LLMs at Content Moderationcited by 27
  5. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  6. 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
  7. 2024VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Valuescited by 12
  8. 2025Vision Language Models as Values Detectorscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
  2. 2026MM-SCALE: Grounded Multimodal Moral Reasoning via Scalar Judgment and Listwise Alignment
  3. 2025Benchmarking Multi-National Value Alignment for Large Language Models
  4. 2024Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models
  5. 2025CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making
  6. 2023Heterogeneous Value Alignment Evaluation for Large Language Models
  7. 2026Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities
  8. 2026ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs