mlsanityData to 14 Oct 2026

Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges

arXiv 2308.00031 ↗First posted 31 Jul 2023cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Generative Artificial Intelligence (AI) is one of the most exciting developments in Computer Science of the last decade. At the same time, Reinforcement Learning (RL) has emerged as a very successful paradigm for a variety of machine learning tasks. In this survey, we discuss the state of the art, opportunities and open research questions in applying RL to generative AI. In particular, we will discuss three types of applications, namely, RL as an alternative way for generation without specified objectives; as a way for generating outputs while concurrently maximizing an objective function; and, finally, as a way of embedding desired characteristics, which cannot be easily captured by means of an objective function, into the generative process. We conclude the survey with an in-depth discussion of the opportunities and challenges in this fascinating emerging area.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 121 of its 170 references that are in the map. First posted on arXiv 31 Jul 2023.

Builds on

121 of 170 references.

References · 121

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2017Proximal Policy Optimization Algorithmscited by 15,215
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2015Continuous control with deep reinforcement learningcited by 3,407
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  12. 2020Learning to summarize from human feedbackcited by 2,588
  13. 2015Trust Region Policy Optimizationcited by 2,580
  14. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  15. 2017Improved Training of Wasserstein GANscited by 2,235
  16. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  17. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  18. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  19. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  20. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  21. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  22. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  23. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  24. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  25. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  26. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  27. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  28. 2015Prioritized Experience Replaycited by 1,099
  29. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  30. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  31. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  32. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  33. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  34. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  35. 2016Self-critical Sequence Training for Image Captioningcited by 547
  36. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  37. 2015A Neural Algorithm of Artistic Stylecited by 483
  38. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450

50 most cited of 121 All 121 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 5; 2026: 2.

10 papers.

  1. 2023User Experience Design Professionals' Perceptions of Generative Artificial Intelligencecited by 16
  2. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  3. 2026Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models?cited by 6
  4. 2025DSO: Direct Steering Optimization for Bias Mitigationcited by 3
  5. 2025Generative Multi-Objective Bayesian Optimization with Scalable Batch Evaluations for Sample-Efficient De Novo Molecular Designcited by 2
  6. 2024A Survey on the Integration of Generative AI for Critical Thinking in Mobile Networkscited by 1
  7. 2025Thinking Outside the (Gray) Box: A Context-Based Score for Assessing Value and Originality in Neural Text Generationcited by 1
  8. 2026On the Creativity of AI Agentscited by 1
  9. 2025Decision Flow Policy Optimizationcited by 0
  10. 2025Enhancing Deep Deterministic Policy Gradients on Continuous Control Tasks with Decoupled Prioritized Experience Replaycited by 0