# Data Generation as Sequential Decision Making

arXiv: https://arxiv.org/abs/1506.03504 · HTML: https://mlsanity.com/paper/1506.03504/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2015.

## Built on by (most cited first, 15 of 15)

- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model](https://mlsanity.com/paper/2311.13231/) (2023)
- [Discrete Probabilistic Inference as Control in Multi-path Environments](https://mlsanity.com/paper/2402.10309/) (2024)
- [AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation](https://mlsanity.com/paper/2409.00342/) (2024)
- [Variational Selective Autoencoder: Learning from Partially-Observed Heterogeneous Data](https://mlsanity.com/paper/2102.12679/) (2021)
- [Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges](https://mlsanity.com/paper/2308.00031/) (2023)
- [Time-series Generation by Contrastive Imitation](https://mlsanity.com/paper/2311.01388/) (2023)
- [Student-Informed Teacher Training](https://mlsanity.com/paper/2412.09149/) (2024)
- [Interpolation of Missing Swaption Volatility Data using Gibbs Sampling on Variational Autoencoders](https://mlsanity.com/paper/2204.10400/) (2022)
- [GANTEE: Generative Adversatial Network for Taxonomy Entering Evaluation](https://mlsanity.com/paper/2303.14480/) (2023)
- [Thinking Outside the (Gray) Box: A Context-Based Score for Assessing Value and Originality in Neural Text Generation](https://mlsanity.com/paper/2502.13207/) (2025)
- [AdaGen: Learning Adaptive Policy for Image Synthesis](https://mlsanity.com/paper/2603.06993/) (2026)
- [Estimating Link Flows in Road Networks with Synthetic Trajectory Data Generation: Reinforcement Learning-based Approaches](https://mlsanity.com/paper/2206.12873/) (2022)
- [HMAMP: Hypervolume-Driven Multi-Objective Antimicrobial Peptides Design](https://mlsanity.com/paper/2405.00753/) (2024)

## Abstract (arXiv)

We connect a broad class of generative models through their shared reliance on sequential decision making. Motivated by this view, we develop extensions to an existing model, and then explore the idea further in the context of data imputation -- perhaps the simplest setting in which to investigate the relation between unconditional and conditional generative modelling. We formulate data imputation as an MDP and develop models capable of representing effective policies for it. We construct the models using neural networks and train them using a form of guided policy search. Our models generate predictions through an iterative process of feedback and refinement. We show that this approach can learn effective policies for imputation problems of varying difficulty and across multiple datasets.