# CrimeMind: Simulating Urban Crime with Multi-Modal LLM Agents

arXiv: https://arxiv.org/abs/2506.05981 · HTML: https://mlsanity.com/paper/2506.05981/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 11 of 11)

- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models](https://mlsanity.com/paper/2501.09686/) (2025)
- [EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities](https://mlsanity.com/paper/2310.10436/) (2023)
- [OpenCity: A Scalable Platform to Simulate Urban Activities with Massive LLM Agents](https://mlsanity.com/paper/2410.21286/) (2024)
- [Emergence of polarization in networks of large language model agents](https://mlsanity.com/paper/2501.05171/) (2025)
- [Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructions](https://mlsanity.com/paper/2408.04168/) (2024)
- [Chain-of-Planned-Behaviour Workflow Elicits Few-Shot Mobility Generation in LLMs](https://mlsanity.com/paper/2402.09836/) (2024)
- [Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network](https://mlsanity.com/paper/2402.11518/) (2024)
- [Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory](https://mlsanity.com/paper/2406.14373/) (2024)
- [AgentSociety Challenge: Designing LLM Agents for User Modeling and Recommendation on Web Platforms](https://mlsanity.com/paper/2502.18754/) (2025)
- [Crime Prediction Using Machine Learning and Deep Learning: A Systematic Review and Future Directions](https://mlsanity.com/paper/2303.16310/) (2023)

## Built on by (most cited first, 2 of 2)

- [When Plausible Is Not Realistic: Evaluating Human Mobility in LLM-Based Urban Simulation](https://mlsanity.com/paper/2606.13835/) (2026)
- [VirtualCrime: Evaluating the Criminal Potential and Agentic Behaviors of Large Language Models via Sandbox Simulation](https://mlsanity.com/paper/2601.13981/) (2026)

## Abstract (arXiv)

Modeling urban crime is an important yet challenging task that requires understanding the subtle visual, social, and cultural cues embedded in urban environments. Previous work has mainly focused on rule-based agent-based modeling (ABM) and deep learning methods. ABMs offer interpretability of internal mechanisms but exhibit limited predictive accuracy. In contrast, deep learning methods are often effective in prediction but are less interpretable and require extensive training data. Moreover, both lines of work lack the cognitive flexibility to adapt to changing environments. Leveraging the capabilities of large language models (LLMs), we propose CrimeMind, a novel LLM-driven ABM framework for simulating urban crime within a multi-modal urban context. A key innovation of our design is the integration of the Routine Activity Theory (RAT) into the agentic workflow of CrimeMind, enabling it to process rich multi-modal urban features and reason about criminal behavior. However, RAT requires LLM agents to infer subtle cues in evaluating environmental safety as part of assessing guardianship, which can be challenging for LLMs. To address this, we collect a small-scale human-annotated dataset and align CrimeMind's perception with human judgment via a training-free textual gradient method. Experiments across four major U.S. cities demonstrate that CrimeMind outperforms both traditional ABMs and deep learning baselines in crime hotspot prediction and spatial distribution accuracy, achieving up to a 24% improvement over the strongest baseline. Furthermore, we conduct counterfactual simulations of external incidents and policy interventions and it successfully captures the expected changes in crime patterns, demonstrating its ability to reflect counterfactual scenarios. Overall, CrimeMind enables fine-grained modeling of individual behaviors and facilitates evaluation of real-world interventions.