# PRISON: Unmasking the Criminal Potential of Large Language Models

arXiv: https://arxiv.org/abs/2506.16150 · HTML: https://mlsanity.com/paper/2506.16150/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 84 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 30 of 51)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark](https://mlsanity.com/paper/2304.03279/) (2023)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://mlsanity.com/paper/2311.07590/) (2023)
- [Evaluating Frontier Models for Dangerous Capabilities](https://mlsanity.com/paper/2403.13793/) (2024)
- [Evaluating and Inducing Personality in Pre-trained Language Models](https://mlsanity.com/paper/2206.07550/) (2022)
- [Explore, Establish, Exploit: Red Teaming Language Models from Scratch](https://mlsanity.com/paper/2306.09442/) (2023)
- [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023)
- [ProAgent: Building Proactive Cooperative Agents with Large Language Models](https://mlsanity.com/paper/2308.11339/) (2023)
- [Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities](https://mlsanity.com/paper/2311.10227/) (2023)
- [Embodied LLM Agents Learn to Cooperate in Organized Teams](https://mlsanity.com/paper/2403.12482/) (2024)
- [Truth is Universal: Robust Detection of Lies in LLMs](https://mlsanity.com/paper/2407.12831/) (2024)
- [AI model GPT-3 (dis)informs us better than humans](https://mlsanity.com/paper/2301.11924/) (2023)
- [International AI Safety Report](https://mlsanity.com/paper/2501.17805/) (2025)
- [On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback](https://mlsanity.com/paper/2411.02306/) (2024)
- [Enhance Reasoning for Large Language Models in the Game Werewolf](https://mlsanity.com/paper/2402.02330/) (2024)
- [CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents](https://mlsanity.com/paper/2401.10568/) (2024)
- [Honesty Is the Best Policy: Defining and Mitigating AI Deception](https://mlsanity.com/paper/2312.01350/) (2023)
- [Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant](https://mlsanity.com/paper/2405.01576/) (2024)
- [Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications](https://mlsanity.com/paper/2411.06837/) (2024)
- [AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents](https://mlsanity.com/paper/2409.09013/) (2024)
- [Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games](https://mlsanity.com/paper/2312.00746/) (2023)
- [Mitigating Deceptive Alignment via Self-Monitoring](https://mlsanity.com/paper/2505.18807/) (2025)

## Built on by (most cited first, 2 of 2)

- [VirtualCrime: Evaluating the Criminal Potential and Agentic Behaviors of Large Language Models via Sandbox Simulation](https://mlsanity.com/paper/2601.13981/) (2026)
- [Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches](https://mlsanity.com/paper/2606.01145/) (2026)

## Abstract (arXiv)

As large language models (LLMs) advance, concerns about their misconduct in complex social contexts intensify. Existing research overlooked the systematic understanding and assessment of their criminal capability in realistic interactions. We propose a unified framework PRISON, to quantify LLMs' criminal potential across five traits: False Statements, Frame-Up, Psychological Manipulation, Emotional Disguise, and Moral Disengagement. Using structured crime scenarios adapted from classic films grounded in reality, we evaluate both criminal potential and anti-crime ability of LLMs. Results show that state-of-the-art LLMs frequently exhibit emergent criminal tendencies, such as proposing misleading statements or evasion tactics, even without explicit instructions. Moreover, when placed in a detective role, models recognize deceptive behavior with only 44% accuracy on average, revealing a striking mismatch between conducting and detecting criminal behavior. These findings underscore the urgent need for adversarial robustness, behavioral alignment, and safety mechanisms before broader LLM deployment.