# AIM-Bench: Evaluating Decision-making Biases of Agentic LLM as Inventory Manager

arXiv: https://arxiv.org/abs/2508.11416 · HTML: https://mlsanity.com/paper/2508.11416/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 9 of 9)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Large language models that replace human participants can harmfully misportray and flatten identity groups](https://mlsanity.com/paper/2402.01908/) (2024)
- [ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation](https://mlsanity.com/paper/2507.04952/) (2025)
- [Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context](https://mlsanity.com/paper/2406.05972/) (2024)
- [MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts](https://mlsanity.com/paper/2502.20808/) (2025)
- [UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios](https://mlsanity.com/paper/2408.17267/) (2024)
- [PokerBench: Training Large Language Models to become Professional Poker Players](https://mlsanity.com/paper/2501.08328/) (2025)
- [Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting](https://mlsanity.com/paper/2405.16133/) (2024)
- [Expand VSR Benchmark for VLLM to Expertize in Spatial Rules](https://mlsanity.com/paper/2412.18224/) (2024)

## Built on by (most cited first, 6 of 6)

- [RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments](https://mlsanity.com/paper/2606.15862/) (2026)
- [OptiRepair: Closed-Loop Diagnosis and Repair of Supply Chain Optimization Models with LLM Agents](https://mlsanity.com/paper/2602.19439/) (2026)
- [RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments](https://mlsanity.com/paper/2603.16453/) (2026)
- [STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle](https://mlsanity.com/paper/2607.13618/) (2026)
- [SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management](https://mlsanity.com/paper/2602.07342/) (2026)
- [CONDESION-BENCH: Conditional Decision-Making of Large Language Models in Compositional Action Space](https://mlsanity.com/paper/2604.09029/) (2026)

## Abstract (arXiv)

Recent advances in mathematical reasoning and the long-term planning capabilities of large language models (LLMs) have precipitated the development of agents, which are being increasingly leveraged in business operations processes. Decision models to optimize inventory levels are one of the core elements of operations management. However, the capabilities of the LLM agent in making inventory decisions in uncertain contexts, as well as the decision-making biases (e.g. framing effect, etc.) of the agent, remain largely unexplored. This prompts concerns regarding the capacity of LLM agents to effectively address real-world problems, as well as the potential implications of biases that may be present. To address this gap, we introduce AIM-Bench, a novel benchmark designed to assess the decision-making behaviour of LLM agents in uncertain supply chain management scenarios through a diverse series of inventory replenishment experiments. Our results reveal that different LLMs typically exhibit varying degrees of decision bias that are similar to those observed in human beings. In addition, we explored strategies to mitigate the pull-to-centre effect and the bullwhip effect, namely cognitive reflection and implementation of information sharing. These findings underscore the need for careful consideration of the potential biases in deploying LLMs in Inventory decision-making scenarios. We hope that these insights will pave the way for mitigating human decision bias and developing human-centred decision support systems for supply chains.