# Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study

arXiv: https://arxiv.org/abs/2403.16517 · HTML: https://mlsanity.com/paper/2403.16517/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 10 of 10)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes](https://mlsanity.com/paper/2312.14890/) (2023)
- [Evaluating Theory of Mind in Question Answering](https://mlsanity.com/paper/1808.09352/) (2018)
- [NormBank: A Knowledge Bank of Situational Social Norms](https://mlsanity.com/paper/2305.17008/) (2023)
- [ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations](https://mlsanity.com/paper/2212.10409/) (2022)
- [Learning Norms from Stories: A Prior for Value Aligned Agents](https://mlsanity.com/paper/1912.03553/) (2019)

## Built on by (most cited first, 6 of 6)

- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies](https://mlsanity.com/paper/2504.19487/) (2025)
- [Can LLMs Reason About Trust?: A Pilot Study](https://mlsanity.com/paper/2507.21075/) (2025)
- [Social Norm Reasoning in Multimodal Language Models: An Evaluation](https://mlsanity.com/paper/2603.03590/) (2026)
- [Evaluating LLM Alignment With Human Trust Models](https://mlsanity.com/paper/2603.05839/) (2026)

## Abstract (arXiv)

Norms are an important component of the social fabric of society by prescribing expected behaviour. In Multi-Agent Systems (MAS), agents interacting within a society are equipped to possess social capabilities such as reasoning about norms and trust. Norms have long been of interest within the Normative Multi-Agent Systems community with researchers studying topics such as norm emergence, norm violation detection and sanctioning. However, these studies have some limitations: they are often limited to simple domains, norms have been represented using a variety of representations with no standard approach emerging, and the symbolic reasoning mechanisms generally used may suffer from a lack of extensibility and robustness. In contrast, Large Language Models (LLMs) offer opportunities to discover and reason about norms across a large range of social situations. This paper evaluates the capability of LLMs to detecting norm violations. Based on simulated data from 80 stories in a household context, with varying complexities, we investigated whether 10 norms are violated. For our evaluations we first obtained the ground truth from three human evaluators for each story. Then, the majority result was compared against the results from three well-known LLM models (Llama 2 7B, Mixtral 7B and ChatGPT-4). Our results show the promise of ChatGPT-4 for detecting norm violations, with Mixtral some distance behind. Also, we identify areas where these models perform poorly and discuss implications for future work.