mlsanityData to 14 Oct 2026

VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs

arXiv 2608.03810 ↗First posted 4 Aug 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models routinely describe socially salient targets, including political figures, countries, religions, organizations, historical events, and social groups, encoding affective framing alongside factual content: a target may appear favorable or threatening, calm or conflictual, powerful or vulnerable. Existing work captures parts of this space through sentiment, favorability, and emotion benchmarks, but none combines target-directed VAD attribution, an explicit scorer contract, and a passport reporting format. We introduce VIBE, a benchmark for entity-centered affective profiling of LLM outputs in Valence-Arousal-Dominance (VAD) space. Its core contribution is a measurement contract: VIBE separates generation from external scoring, distinguishes scalar favorability, response-level VAD, and target-directed VAD, and reports profiles through an Affective Passport. Three empirical layers support the contract. H1 shows scalar favorability does not subsume arousal and dominance: valence findings are cross-validated (rV = 0.944 judge-human, rV = 0.954 inter-scorer); arousal and dominance are single-scorer directional estimates, not point-precise, consistent with known inter-annotator difficulty on these axes (rA = 0.495, rD = 0.702 among human annotators). H2 shows whole-response and target-directed VAD are different contracts: the same text can carry one affective tone overall while representing the named target differently. H3 is a protocol-drift diagnostic: elicitation conditions shift profiles, motivating context metadata in every affective report. These results motivate entity-centered affective profiling as a documented practice: profiles should be released with scorer identity, coverage, protocol, and interpretation limits.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

25 of 41 references.

References · 25

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2018Datasheets for Datasetscited by 1,386
  3. 2018Model Cards for Model Reportingcited by 1,080
  4. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  5. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  6. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  7. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  8. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  9. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  10. 2019Measurement and Fairnesscited by 308
  11. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
  12. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
13 more
  1. 2024Quantifying the Persona Effect in LLM Simulationscited by 169
  2. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
  3. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  4. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  5. 2022EmoBank: Studying the Impact of Annotation Perspective and Representation Format on Dimensional Emotion Analysiscited by 76
  6. 2024POSIX: A Prompt Sensitivity Index For Large Language Modelscited by 59
  7. 2024Are Large Language Models Consistent over Value-laden Questions?cited by 58
  8. 2024Large Language Models Reflect the Ideology of their Creatorscited by 57
  9. 2017WASSA-2017 Shared Task on Emotion Intensitycited by 38
  10. 2025Social Bias Benchmark for Generation: A Comparison of Generation and QA-Based Evaluationscited by 16
  11. 2025Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Modelscited by 15
  12. 2023Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformerscited by 11
  13. 2025SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detectioncited by 9

Built on by

Not cited yet.