Validating Generative Agent-Based Models for Logistics and Supply Chain Management Research
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Generative Agent-Based Models (GABMs) powered by large language models (LLMs) offer promising potential for empirical logistics and supply chain management (LSCM) research by enabling realistic simulation of complex human behaviors. Unlike traditional agent-based models, GABMs generate human-like responses through natural language reasoning, which creates potential for new perspectives on emergent LSCM phenomena. However, the validity of LLMs as proxies for human behavior in LSCM simulations is unknown. This study evaluates LLM equivalence of human behavior through a controlled experiment examining dyadic customer-worker engagements in food delivery scenarios. I test six state-of-the-art LLMs against 957 human participants (477 dyads) using a moderated mediation design. This study reveals a need to validate GABMs on two levels: (1) human equivalence testing, and (2) decision process validation. Results reveal GABMs can effectively simulate human behaviors in LSCM; however, an equivalence-versus-process paradox emerges. While a series of Two One-Sided Tests (TOST) for equivalence reveals some LLMs demonstrate surface-level equivalence to humans, structural equation modeling (SEM) reveals artificial decision processes not present in human participants for some LLMs. These findings show GABMs as a potentially viable methodological instrument in LSCM with proper validation checks. The dual-validation framework also provides LSCM researchers with a guide to rigorous GABM development. For practitioners, this study offers evidence-based assessment for LLM selection for operational tasks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 9 of its 101 references that are in the map. First posted on arXiv 27 Aug 2025.
Builds on
9 of 101 references.
Closest ancestors · 5
Same direction, newest first
- 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations4 Apr
- 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia6 Dec
- 2023Simulating Public Administration Crisis: A Novel Generative Agent-Based Simulation System to Lower Technology Barriers in Social Science Research12 Nov
- 2023Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering14 Aug
- 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr
Other references · 4
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
- 2024Toward LLM-Agent-Based Modeling of Transportation Systems: A Conceptual Frameworkcited by 19
- 2025Large Language Models for Supply Chain Decisionscited by 13
- 2023Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulationscited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Predicting Effects, Missing Distributions: Evaluating LLMs as Human Behavior Simulators in Operations Management
- 2024Agentic LLMs in the Supply Chain: Towards Autonomous Multi-Agent Consensus-Seeking
- 2026Dynamics of Cognitive Heterogeneity: Investigating Behavioral Biases in Multi-Stage Supply Chains with LLM-Based Simulation
- 2023Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence
- 2026Interpreting and Steering LLM Agents for Social Simulations
- 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
- 2025AIM-Bench: Evaluating Decision-making Biases of Agentic LLM as Inventory Manager
- 2025Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-Making