# Approximate information for efficient exploration-exploitation strategies

arXiv: https://arxiv.org/abs/2307.01563 · HTML: https://mlsanity.com/paper/2307.01563/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 28 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 4 of 4)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [A Survey on Contextual Multi-armed Bandits](https://mlsanity.com/paper/1508.03326/) (2015)
- [An empirical evaluation of active inference in multi-armed bandits](https://mlsanity.com/paper/2101.08699/) (2021)
- [Infomax strategies for an optimal balance between exploration and exploitation](https://mlsanity.com/paper/1601.03073/) (2016)

## Built on by (most cited first, 2 of 2)

- ["Well, Keep Thinking": Enhancing LLM Reasoning with Adaptive Injection Decoding](https://mlsanity.com/paper/2503.10167/) (2025)
- [Approximate information maximization for bandit games](https://mlsanity.com/paper/2310.12563/) (2023)

## Abstract (arXiv)

This paper addresses the exploration-exploitation dilemma inherent in decision-making, focusing on multi-armed bandit problems. The problems involve an agent deciding whether to exploit current knowledge for immediate gains or explore new avenues for potential long-term rewards. We here introduce a novel algorithm, approximate information maximization (AIM), which employs an analytical approximation of the entropy gradient to choose which arm to pull at each point in time. AIM matches the performance of Infomax and Thompson sampling while also offering enhanced computational speed, determinism, and tractability. Empirical evaluation of AIM indicates its compliance with the Lai-Robbins asymptotic bound and demonstrates its robustness for a range of priors. Its expression is tunable, which allows for specific optimization in various settings.