mlsanityData to 14 Oct 2026

Reinforcement Learning for Generative AI: A Survey

arXiv 2308.14328 ↗First posted 28 Aug 2023cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Deep Generative AI has been a long-standing essential topic in the machine learning community, which can impact a number of application areas like text generation and computer vision. The major paradigm to train a generative model is maximum likelihood estimation, which pushes the learner to capture and approximate the target data distribution by decreasing the divergence between the model distribution and the target distribution. This formulation successfully establishes the objective of generative tasks, while it is incapable of satisfying all the requirements that a user might expect from a generative model. Reinforcement learning, serving as a competitive option to inject new training signals by creating new objectives that exploit novel signals, has demonstrated its power and flexibility to incorporate human inductive bias from multiple angles, such as adversarial learning, hand-designed rules and learned reward model to build a performant model. Thereby, reinforcement learning has become a trending research field and has stretched the limits of generative AI in both model design and application. It is reasonable to summarize and conclude advances in recent years with a comprehensive review. Although there are surveys in different application areas recently, this survey aims to shed light on a high-level review that spans a range of application areas. We provide a rigorous taxonomy in this area and make sufficient coverage on various models and applications. Notably, we also surveyed the fast-developing large language model area. We conclude this survey by showing the potential directions that might tackle the limit of current models and expand the frontiers for generative AI.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 179 of its 306 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

179 of 306 references.

References · 179

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2017Proximal Policy Optimization Algorithmscited by 15,215
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  11. 2015Continuous control with deep reinforcement learningcited by 3,407
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
38 more
  1. 2020Learning to summarize from human feedbackcited by 2,588
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2015Variational Inference with Normalizing Flowscited by 2,170
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  9. 2016OpenAI Gymcited by 1,736
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  11. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  12. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  13. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  14. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  15. 2016Pixel Recurrent Neural Networkscited by 816
  16. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  17. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  18. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  19. 2016Self-critical Sequence Training for Image Captioningcited by 547
  20. 2022Efficiently Scaling Transformer Inferencecited by 539
  21. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 521
  22. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  23. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  24. 2018MolGAN: An implicit generative model for small molecular graphscited by 393
  25. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 376
  26. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  27. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 365
  28. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  29. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  30. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  31. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  32. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  33. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  34. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  35. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  36. 2021How to Train Your Energy-Based Modelscited by 274
  37. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
  38. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247

50 most cited of 179 All 179 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 1.

11 papers.

  1. 2025Improving Medical Large Vision-Language Models with Abnormal-Aware Feedbackcited by 42
  2. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Modelscited by 15
  3. 2025Generative Models in Decision Making: A Surveycited by 15
  4. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 6
  5. 2025Expand and Prune: Maximizing Trajectory Diversity for Effective GRPO in Generative Modelscited by 6
  6. 2025Learning Intractable Multimodal Policies with Reparameterization and Diversity Regularizationcited by 3
  7. 2024The Superalignment of Superhuman Intelligence with Large Language Modelscited by 2
  8. 2025Online Robust Reinforcement Learning with General Function Approximationcited by 2
  9. 2024Automating Creativitycited by 0
  10. 2025Decision Flow Policy Optimizationcited by 0
  11. 2026Diffusion Offline Reinforcement Learning for Fair and Energy-Efficient UAV-Assisted Wireless Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances
  2. 2024Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective
  3. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Review
  4. 2025Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
  5. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations
  6. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions
  7. 2024Reinforcement Learning with Generative Models for Compact Support Sets
  8. 2025Generative AI for Vision: A Comprehensive Study of Frameworks and Applications