# A Behavioral Model for Exploration vs. Exploitation: Theoretical Framework and Experimental Evidence

arXiv: https://arxiv.org/abs/2207.01028 · HTML: https://mlsanity.com/paper/2207.01028/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2022.

## Built on by (most cited first, 2 of 2)

- [Do LLM Agents Have Regret? A Case Study in Online Learning and Games](https://mlsanity.com/paper/2403.16843/) (2024)
- [Comparing Exploration-Exploitation Strategies of LLMs and Humans: Insights from Standard Multi-armed Bandit Experiments](https://mlsanity.com/paper/2505.09901/) (2025)

## Abstract (arXiv)

How do people navigate the exploration-exploitation (EE) trade-off when making repeated choices with unknown rewards? We study this question through the lens of multi-armed bandit problems and introduce a novel behavioral model, Quantal Choice with Adaptive Reduction of Exploration (QCARE). It generalizes Thompson Sampling, allowing for a principled way to quantify the EE trade-off and reflect human decision-making patterns. The model adaptively reduces exploration as information accumulates, with the reduction rate serving as a parameter to quantify the EE trade-off dynamics. We theoretically analyze how varying reduction rates influence decision quality, shedding light on the effects of ``over-exploration'' and ``under-exploration.'' Empirically, we validate QCARE through experiments collecting behavioral data from human participants. QCARE not only captures critical behavioral patterns in the EE trade-off but also outperforms alternative models in predictive power. Our analysis reveals a behavioral tendency toward over-exploration.