# On Exploration, Exploitation and Learning in Adaptive Importance Sampling

arXiv: https://arxiv.org/abs/1810.13296 · HTML: https://mlsanity.com/paper/1810.13296/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 1 of 1)

- [Multi-Armed Sampling Problem and the End of Exploration](https://mlsanity.com/paper/2507.10797/) (2025)

## Abstract (arXiv)

We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation. Borrowing ideas from the bandits literature, we propose Daisee, a partition-based AIS algorithm. We further introduce a notion of regret for AIS and show that Daisee has O(√T(log T)^frac34) cumulative pseudo-regret, where T is the number of iterations. We then extend Daisee to adaptively learn a hierarchical partitioning of the sample space for more efficient sampling and confirm the performance of both algorithms empirically.