# How Different AI Chatbots Behave? Benchmarking Large Language Models in Behavioral Economics Games

arXiv: https://arxiv.org/abs/2412.12362 · HTML: https://mlsanity.com/paper/2412.12362/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 3 of 3)

- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [Can Large Language Model Agents Simulate Human Trust Behavior?](https://mlsanity.com/paper/2402.04559/) (2024)

## Built on by (most cited first, 6 of 6)

- [A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives](https://mlsanity.com/paper/2503.13415/) (2025)
- [A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis](https://mlsanity.com/paper/2502.16879/) (2025)
- [Be.FM: Open Foundation Models for Human Behavior](https://mlsanity.com/paper/2505.23058/) (2025)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks](https://mlsanity.com/paper/2606.24162/) (2026)
- [Serious Games: Human-AI Interaction, Evolution, and Coevolution](https://mlsanity.com/paper/2505.16388/) (2025)

## Abstract (arXiv)

The deployment of large language models (LLMs) in diverse applications requires a thorough understanding of their decision-making strategies and behavioral patterns. As a supplement to a recent study on the behavioral Turing test, this paper presents a comprehensive analysis of five leading LLM-based chatbot families as they navigate a series of behavioral economics games. By benchmarking these AI chatbots, we aim to uncover and document both common and distinct behavioral patterns across a range of scenarios. The findings provide valuable insights into the strategic preferences of each LLM, highlighting potential implications for their deployment in critical decision-making roles.