mlsanityData to 14 Oct 2026

Value-Spectrum: Quantifying Preferences of Vision-Language Models via Value Decomposition in Social Media Contexts

arXiv 2411.11479 ↗First posted 18 Nov 2024cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The recent progress in Vision-Language Models (VLMs) has broadened the scope of multimodal applications. However, evaluations often remain limited to functional tasks, neglecting abstract dimensions such as personality traits and human values. To address this gap, we introduce Value-Spectrum, a novel Visual Question Answering (VQA) benchmark aimed at assessing VLMs based on Schwartz's value dimensions that capture core human values guiding people's preferences and actions. We design a VLM agent pipeline to simulate video browsing and construct a vector database comprising over 50,000 short videos from TikTok, YouTube Shorts, and Instagram Reels. These videos span multiple months and cover diverse topics, including family, health, hobbies, society, technology, etc. Benchmarking on Value-Spectrum highlights notable variations in how VLMs handle value-oriented content. Beyond identifying VLMs' intrinsic preferences, we also explore the ability of VLM agents to adopt specific personas when explicitly prompted, revealing insights into the adaptability of the model in role-playing scenarios. These findings highlight the potential of Value-Spectrum as a comprehensive evaluation set for tracking VLM preferences in value-based tasks and abilities to simulate diverse personas. The complete code and data are available at: https://github.com/Jeremyyny/Value-Spectrum.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 45 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

20 of 45 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024CogVLM2: Visual Language Models for Image and Video Understanding29 Aug
  2. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences16 Jun
  3. 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions19 Aug
  4. 2023Vision-Language Models for Vision Tasks: A Survey3 Apr

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  3. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  4. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  5. 2023Role-Play with Large Language Modelscited by 294
  6. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
  7. 2023InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviewscited by 179
  8. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  9. 2024ScreenAgent: A Vision Language Model-driven Computer Control Agentcited by 102
  10. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  11. 2023On the steerability of large language models toward data-driven personascited by 53
  12. 2023Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systemscited by 45
4 more
  1. 2023Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusioncited by 21
  2. 2024The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approachescited by 18
  3. 2021SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialoguescited by 9
  4. 2020Adjusted chi-square test for degree-corrected block modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities
  2. 2025SPECTRA: Revealing the Full Spectrum of User Preferences via Distributional LLM Inference
  3. 2026A Dataset for Dynamic Human Preferences for Vision Language Models
  4. 2026ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs
  5. 2025Survey-to-Behavior: Downstream Alignment of Human Values in LLMs via Survey Questions
  6. 2025Deep Value Benchmark: Measuring Whether Models Generalize Deep Values or Shallow Preferences
  7. 2025A Video Is Not Worth a Thousand Words
  8. 2024VisionArena: 230K Real World User-VLM Conversations with Preference Labels