# Social Norm Reasoning in Multimodal Language Models: An Evaluation

arXiv: https://arxiv.org/abs/2603.03590 · HTML: https://mlsanity.com/paper/2603.03590/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [Emergence of Social Norms in Generative Agent Societies: Principles and Architecture](https://mlsanity.com/paper/2403.08251/) (2024)
- [Measuring Social Norms of Large Language Models](https://mlsanity.com/paper/2404.02491/) (2024)
- [Agent Alignment in Evolving Social Norms](https://mlsanity.com/paper/2401.04620/) (2024)
- [Evolution of Social Norms in LLM Agents using Natural Language](https://mlsanity.com/paper/2409.00993/) (2024)
- [EgoNormia: Benchmarking Physical Social Norm Understanding](https://mlsanity.com/paper/2502.20490/) (2025)
- [Harnessing the power of LLMs for normative reasoning in MASs](https://mlsanity.com/paper/2403.16524/) (2024)
- [Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study](https://mlsanity.com/paper/2403.16517/) (2024)
- [Can LLMs Understand Social Norms in Autonomous Driving Games?](https://mlsanity.com/paper/2408.12680/) (2024)
- [Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies](https://mlsanity.com/paper/2504.19487/) (2025)
- [Extracting Norms from Contracts Via ChatGPT: Opportunities and Challenges](https://mlsanity.com/paper/2404.02269/) (2024)

## Built on by (most cited first, 1 of 1)

- [NormAct: Benchmarking Embodied Agents' Proactive Compliance with Unspoken Social Norms](https://mlsanity.com/paper/2606.27826/) (2026)

## Abstract (arXiv)

In Multi-Agent Systems (MAS), agents are designed with social capabilities, allowing them to understand and reason about social concepts such as norms when interacting with others (e.g., inter-robot interactions). In Normative MAS (NorMAS), researchers study how norms develop, and how violations are detected and sanctioned. However, existing research in NorMAS use symbolic approaches (e.g., formal logic) for norm representation and reasoning whose application is limited to simplified environments. In contrast, Multimodal Large Language Models (MLLMs) present promising possibilities to develop software used by robots to identify and reason about norms in a wide variety of complex social situations embodied in text and images. However, prior work on norm reasoning have been limited to text-based scenarios. This paper investigates the norm reasoning competence of five MLLMs by evaluating their ability to answer norm-related questions based on thirty text-based and thirty image-based stories, and comparing their responses against humans. Our results show that MLLMs demonstrate superior performance in norm reasoning in text than in images. GPT-4o performs the best in both modalities offering the most promise for integration with MAS, followed by the free model Qwen-2.5VL. Additionally, all models find reasoning about complex norms challenging.