# Can Large Language Models Implement Agent-Based Models? An ODD-based Replication Study

arXiv: https://arxiv.org/abs/2602.10140 · HTML: https://mlsanity.com/paper/2602.10140/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 65 references that are in the map.
- First posted on arXiv 8 Feb 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence](https://mlsanity.com/paper/2406.11931/) (2024)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [CodeGemma: Open Code Models Based on Gemma](https://mlsanity.com/paper/2406.11409/) (2024)
- [LLM4SR: A Survey on Large Language Models for Scientific Research](https://mlsanity.com/paper/2501.04306/) (2025)
- [MaintainCoder: Maintainable Code Generation Under Dynamic Requirements](https://mlsanity.com/paper/2503.24260/) (2025)
- [Leveraging Large Language Models for Code Translation and Software Development in Scientific Computing](https://mlsanity.com/paper/2410.24119/) (2024)
- [hyppo: A Multivariate Hypothesis Testing Python Package](https://mlsanity.com/paper/1907.02088/) (2019)
- [GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries](https://mlsanity.com/paper/2508.00033/) (2025)
- [Assessing the Quality and Security of AI-Generated Code: A Quantitative Analysis](https://mlsanity.com/paper/2508.14727/) (2025)
- [Do Large Language Models Speak Scientific Workflows?](https://mlsanity.com/paper/2412.10606/) (2024)
- [Dynamic Stability of LLM-Generated Code](https://mlsanity.com/paper/2511.07463/) (2025)
- [SIMCODE: A Benchmark for Natural Language to ns-3 Network Simulation Code Generation](https://mlsanity.com/paper/2507.11014/) (2025)
- [Parallelization Strategies for Spatial Agent-Based Models](https://mlsanity.com/paper/1507.04047/) (2015)

## Built on by (most cited first, 1 of 1)

- [AIvilization v0: Toward Large-Scale Artificial Social Simulation with a Unified Agent Architecture and Adaptive Agent Profiles](https://mlsanity.com/paper/2602.10429/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can now synthesize non-trivial executable code from textual descriptions, raising an important question: can LLMs reliably implement agent-based models from standardized specifications in a way that supports replication, verification, and validation? We address this question by evaluating 17 contemporary LLMs on a controlled ODD-to-code translation task, using the PPHPC predator-prey model as a fully specified reference. Generated Python implementations are assessed through staged executability checks, model-independent statistical comparison against a validated NetLogo baseline, and quantitative measures of runtime efficiency and maintainability. Results show that behaviorally faithful implementations are achievable but not guaranteed, and that executability alone is insufficient for scientific use. GPT-4.1 consistently produces statistically valid and efficient implementations, with Claude 3.7 Sonnet performing well but less reliably. Overall, the findings clarify both the promise and current limitations of LLMs as model engineering tools, with implications for reproducible agent-based and ecological modeling.