# VideoNorms: Benchmarking Cultural Awareness of Video Language Models

arXiv: https://arxiv.org/abs/2510.08543 · HTML: https://mlsanity.com/paper/2510.08543/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 80 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 30 of 36)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering](https://mlsanity.com/paper/1704.04497/) (2017)
- [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://mlsanity.com/paper/2005.00333/) (2020)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2504.06958/) (2025)
- [Can language models learn from explanations in context?](https://mlsanity.com/paper/2204.02329/) (2022)
- [On Releasing Annotator-Level Labels and Information in Datasets](https://mlsanity.com/paper/2110.05699/) (2021)
- [CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies](https://mlsanity.com/paper/2404.15238/) (2024)
- [FLUTE: Figurative Language Understanding through Textual Explanations](https://mlsanity.com/paper/2205.12404/) (2022)
- [Extracting Cultural Commonsense Knowledge at Scale](https://mlsanity.com/paper/2210.07763/) (2022)
- [MovieGraphs: Towards Understanding Human-Centric Situations from Videos](https://mlsanity.com/paper/1712.06761/) (2017)
- [NormBank: A Knowledge Bank of Situational Social Norms](https://mlsanity.com/paper/2305.17008/) (2023)
- [Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information](https://mlsanity.com/paper/2301.05036/) (2023)
- [NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly](https://mlsanity.com/paper/2210.08604/) (2022)
- [V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding](https://mlsanity.com/paper/2412.09616/) (2024)
- [Social Genome: Grounded Social Reasoning Abilities of Multimodal Models](https://mlsanity.com/paper/2502.15109/) (2025)
- [NormDial: A Comparable Bilingual Synthetic Dialog Dataset for Modeling Social Norm Adherence and Violation](https://mlsanity.com/paper/2310.14563/) (2023)
- [Unifying Data Perspectivism and Personalization: An Application to Social Norms](https://mlsanity.com/paper/2210.14531/) (2022)

## Built on by (most cited first, 4 of 4)

- [Computational Hermeneutics: Evaluating generative AI as a cultural technology](https://mlsanity.com/paper/2604.16403/) (2026)
- [CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generation](https://mlsanity.com/paper/2608.01942/) (2026)
- [Quantifying the Salience of Geo-Cultural Values for Pluralistic Safety Alignment](https://mlsanity.com/paper/2606.00369/) (2026)
- [Cultural Moment Benchmark: Evaluating Video Cultural Reasoning and Grounding in Southeast Asia](https://mlsanity.com/paper/2608.23065/) (2026)

## Abstract (arXiv)

As Video Large Language Models (VideoLLMs) are deployed globally, it is important to assess their ability to reason across cultural contexts. To advance cultural norm awareness evaluation in VideoLLMs, we introduce VideoNorms, a dataset of cultural norm annotations from popular US and Chinese TV shows annotated with adherence or violation labels and (non-)verbal evidence. Through a human-AI collaboration framework, each item was first annotated by a large VideoLLM, and then reviewed by at least three trained monocultural annotators with significant lived experience in the target culture, resulting in a dataset of over 3,000 human judgments. Human verification showed disparity in US and Chinese norm extraction performance, cautioning against fully automatic approaches cultures under-represented in training data. Hierarchical linear modeling analysis of 7 open-weight VideoLLMs' performance revealed that: 1) models perform worse in Chinese compared to US, particularly for norm adherence prediction; 2) models have more difficulty in providing non-verbal evidence compared to verbal evidence for norm adherence/violation predictions. Ablation studies confirm video modality is indeed necessary for accurate performance, and scaling model size does not yield classification score improvements. Our findings and data contribute to culturally grounded video model training and evaluation.