mlsanityData to 14 Oct 2026

The Duality of Generative AI and Reinforcement Learning in Robotics: A Review

arXiv 2410.16411 ↗First posted 21 Oct 2024cs.RO cs.LGcited by 6

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Recently, generative AI and reinforcement learning (RL) have been redefining what is possible for AI agents that take information flows as input and produce intelligent behavior. As a result, we are seeing similar advancements in embodied AI and robotics for control policy generation. Our review paper examines the integration of generative AI models with RL to advance robotics. Our primary focus is on the duality between generative AI and RL for robotics downstream tasks. Specifically, we investigate: (1) The role of prominent generative AI tools as modular priors for multi-modal input fusion in RL tasks. (2) How RL can train, fine-tune and distill generative models for policy generation, such as VLA models, similarly to RL applications in large language models. We then propose a new taxonomy based on a considerable amount of selected papers. Lastly, we identify open challenges accounting for model scalability, adaptation and grounding, giving recommendations and insights on future research directions. We reflect on which generative AI models best fit the RL tasks and why. On the other side, we reflect on important issues inherent to RL-enhanced generative policies, such as safety concerns and failure modes, and what are the limitations of current methods. A curated collection of relevant research papers is maintained on our GitHub repository, serving as a resource for ongoing research and development in this field: https://github.com/clmoro/Robotics-RL-FMs-Integration.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 146 of its 171 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

146 of 171 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuning25 Sep
  2. 2024Affordance-Guided Reinforcement Learning via Visual Prompting14 Jul
  3. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Models24 May
  4. 2024DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets30 Apr

Other references · 142

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Classifier-Free Diffusion Guidancecited by 5,847
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  9. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
38 more
  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  5. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  6. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  7. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  8. 2022A Generalist Agentcited by 808
  9. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  10. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  11. 2024Genie: Generative Interactive Environmentscited by 747
  12. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  13. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  14. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  15. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  16. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  17. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  18. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  19. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  20. 2023Learning Interactive Real-World Simulatorscited by 458
  21. 2015Policy Distillationcited by 440
  22. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  23. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  24. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  25. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  26. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  27. 2024Diffusion Policy Policy Optimizationcited by 280
  28. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  29. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  30. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
  31. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 216
  32. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  33. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  34. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  35. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  36. 2024An Introduction to Vision-Language Modelingcited by 178
  37. 2022Masked World Models for Visual Controlcited by 170
  38. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 165

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  2. 2025Generative AI for Autonomous Driving: Frontiers and Opportunitiescited by 7
  3. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agentscited by 1
  4. 2026RFM-Pose:Reinforcement-Guided Flow Matching for Fast Category-Level 6D Pose Estimationcited by 0
  5. 2026Diffusion Sequence Models for Generative In-Context Meta-Learning of Robot Dynamicscited by 0
  6. 2026Stabilizer Design for Policy Iteration in Stochastic Linear Quadratic Control: A Spectrum-Assignment Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative Models in Decision Making: A Survey
  2. 2025GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspective
  3. 2026A Comprehensive Review of Generative Physical Artificial Intelligence
  4. 2024Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective
  5. 2023Reinforcement Learning for Generative AI: A Survey
  6. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
  7. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Control
  8. 2026RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process