# Active Exploration via Autoregressive Generation of Missing Data

arXiv: https://arxiv.org/abs/2405.19466 · HTML: https://mlsanity.com/paper/2405.19466/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2024.

## Built on by (most cited first, 9 of 9)

- [Contextual Thompson Sampling via Generation of Missing Data](https://mlsanity.com/paper/2502.07064/) (2025)
- [Learning-To-Measure: In-Context Active Feature Acquisition](https://mlsanity.com/paper/2510.12624/) (2025)
- [Pre-Trained AI Model Assisted Online Decision-Making under Missing Covariates: A Theoretical Perspective](https://mlsanity.com/paper/2507.07852/) (2025)
- [OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents](https://mlsanity.com/paper/2605.11169/) (2026)
- [Reasonably reasoning AI agents can avoid game-theoretic failures in zero-shot, provably](https://mlsanity.com/paper/2603.18563/) (2026)
- [Equitable Evaluation via Elicitation](https://mlsanity.com/paper/2602.21327/) (2026)
- [In-Context Learning for Data-Driven Censored Inventory Control](https://mlsanity.com/paper/2605.14840/) (2026)
- [Are Good Generators Good Decision-Makers? Policy Learning for General Interventions via Retargeted Counterfactual Generation](https://mlsanity.com/paper/2606.07399/) (2026)
- [Bootstrap-Conditioned Action Selection with Tabular Foundation Models](https://mlsanity.com/paper/2608.06559/) (2026)

## Abstract (arXiv)

We pose uncertainty quantification and exploration in online decision-making as a problem of training and generation from an autoregressive sequence model, an area experiencing rapid innovation. Our approach rests on viewing uncertainty as arising from missing future outcomes that could be revealed through action choices, rather than from unobservable latent parameters of the environment. This reformulation aligns naturally with modern machine learning capabilities: we can i) train generative models through next-outcome prediction rather than fit explicit priors, ii) assess uncertainty through autoregressive generation rather than sampling latent parameters from posteriors, and iii) adapt to new information by extending the sequence model's context rather than explicit posterior updating. Our main theoretical result establishes a reduction from online decision-making to offline next-outcome prediction: Bayesian regret is controlled directly by the sequence model's offline prediction loss, without requiring an explicit latent-variable posterior. Experiments, including a semi-synthetic news recommendation task, show that autoregressive generation produces calibrated epistemic uncertainty and enables effective exploration by using article text as prior information to focus exploration on resolving remaining uncertainties.