# Generative Models in Decision Making: A Survey

arXiv: https://arxiv.org/abs/2502.17100 · HTML: https://mlsanity.com/paper/2502.17100/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 249 of its 334 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 30 of 249)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)

## Built on by (most cited first, 15 of 15)

- [UltraVSR: Achieving Ultra-Realistic Video Super-Resolution with Efficient One-Step Diffusion Space](https://mlsanity.com/paper/2505.19958/) (2025)
- [PUMA: Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning](https://mlsanity.com/paper/2507.08064/) (2025)
- [Conditioning Matters: Training Diffusion Policies is Faster Than You Think](https://mlsanity.com/paper/2505.11123/) (2025)
- [Two-Steps Diffusion Policy for Robotic Manipulation via Genetic Denoising](https://mlsanity.com/paper/2510.21991/) (2025)
- [Q-Flow: Stable and Expressive Reinforcement Learning with Flow-Based Policy](https://mlsanity.com/paper/2605.13435/) (2026)
- [EmbodiedMAE: A Unified 3D Multi-Modal Representation for Robot Manipulation](https://mlsanity.com/paper/2505.10105/) (2025)
- [FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies](https://mlsanity.com/paper/2603.27450/) (2026)
- [VideoGAN-based Trajectory Proposal for Automated Vehicles](https://mlsanity.com/paper/2506.16209/) (2025)
- [Enhancing Generative Auto-bidding with Offline Reward Evaluation and Policy Search](https://mlsanity.com/paper/2509.15927/) (2025)
- [KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation](https://mlsanity.com/paper/2607.04652/) (2026)
- [Decision Flow Policy Optimization](https://mlsanity.com/paper/2505.20350/) (2025)
- [Sampler-Robust Optimization under Generative Models](https://mlsanity.com/paper/2604.27447/) (2026)
- [Model-Based Proactive Cost Generation for Learning Safe Policies Offline with Limited Violation Data](https://mlsanity.com/paper/2605.01356/) (2026)
- [Extended Field of View Analysis for VideoGAN-based Trajectory Generation](https://mlsanity.com/paper/2608.02289/) (2026)
- [OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation](https://mlsanity.com/paper/2610.15414/) (2026)

## Abstract (arXiv)

Generative models have fundamentally reshaped the landscape of decision-making, reframing the problem from pure scalar reward maximization to high-fidelity trajectory generation and distribution matching. This paradigm shift addresses intrinsic limitations in classical Reinforcement Learning (RL), particularly the limited expressivity of standard unimodal policy distributions in capturing complex, multi-modal behaviors embedded in diverse datasets. However, current literature often treats these models as isolated algorithmic improvements, rarely synthesizing them into a single comprehensive framework. This survey proposes a principled taxonomy grounding generative decision-making within the probabilistic framework of Control as Inference. By performing a variational factorization of the trajectory posterior, we conceptualize four distinct functional roles: Controllers for amortized policy inference, Modelers for dynamics priors, Optimizers for iterative trajectory refinement, and Evaluators for trajectory guidance and value assessment. Unlike existing architecture-centric reviews, this function-centric framework allows us to critically analyze representative generative families across distinct dimensions. Furthermore, we examine deployment in high-stakes domains, specifically Embodied AI, Autonomous Driving, and AI for Science, highlighting systemic risks such as dynamics hallucination in world models and proxy exploitation. Finally, we chart the path toward Generalist Physical Intelligence, identifying pivotal challenges in inference efficiency, trustworthiness, and the emergence of Physical Foundation Models.