mlsanityData to 7 Oct 2026

Model-Based Offline Planning

arXiv 2008.05556 ↗First posted 12 Aug 2020cs.LG cs.AI cs.ROcited by 126

Not on the map. Why?

Cited by 80 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Offline learning is a key part of making reinforcement learning (RL) useable in real systems. Offline RL looks at scenarios where there is data from a system's operation, but no direct access to the system when learning a policy. Recent work on training RL policies from offline data has shown results both with model-free policies learned directly from the data, or with planning on top of learnt models of the data. Model-free policies tend to be more performant, but are more opaque, harder to command externally, and less easy to integrate into larger systems. We propose an offline learner that generates a model that can be used to control the system directly through planning. This allows us to have easily controllable policies directly from data, without ever interacting with the system. We show the performance of our algorithm, Model-Based Offline Planning (MBOP) on a series of robotics-inspired tasks, and demonstrate its ability leverage planning to respect environmental constraints. We are able to find near-optimal polices for certain simulated systems from as little as 50 seconds of real-time system interaction, and create zero-shot goal-conditioned policies on a series of environments. An accompanying video can be found here: https://youtu.be/nxGGHdZOFts

Key facts. Cited by 126 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 26; 2022: 29; 2023: 30; 2024: 18; 2025: 14; 2026: 8.

126 papers.

Sort:

By area:

  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  3. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  4. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  5. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  6. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  7. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Plannerscited by 147
  8. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  9. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  10. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  11. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  12. 2023Elastic Decision Transformercited by 66
38 more
  1. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  2. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  3. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  4. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  5. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  6. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  7. 2022Skill-based Model-based Reinforcement Learningcited by 50
  8. 2021DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learningcited by 47
  9. 2021Vector Quantized Models for Planningcited by 47
  10. 2021Offline Reinforcement Learning with Reverse Model-based Imaginationcited by 47
  11. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  12. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
  13. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  14. 2021Offline Reinforcement Learning with Value-based Episodic Memorycited by 36
  15. 2024Diffusion Model Predictive Controlcited by 35
  16. 2022Adaptive Behavior Cloning Regularization for Stable Offline-to-Online Reinforcement Learningcited by 34
  17. 2023World Models via Policy-Guided Trajectory Diffusioncited by 34
  18. 2021Model-Based Offline Planning with Trajectory Pruningcited by 30
  19. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
  20. 2021Residual Reinforcement Learning from Demonstrationscited by 27
  21. 2023Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Planscited by 27
  22. 2021Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RLcited by 24
  23. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planningcited by 23
  24. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  25. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targetscited by 22
  26. 2024DNAct: Diffusion Guided Multi-Task 3D Policy Learningcited by 22
  27. 2022Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imaginationcited by 21
  28. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
  29. 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21
  30. 2022Mastering the Unsupervised Reinforcement Learning Benchmark from Pixelscited by 20
  31. 2022Improving TD3-BC: Relaxed Policy Constraint for Offline Learning and Stable Online Fine-Tuningcited by 20
  32. 2023Graph Decision Transformercited by 20
  33. 2023ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensemblescited by 20
  34. 2023GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Modelscited by 20
  35. 2025floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RLcited by 20
  36. 2021Towards Deeper Deep Reinforcement Learning with Spectral Normalizationcited by 19
  37. 2023Gradient-based Planning with World Modelscited by 18
  38. 2023Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matchingcited by 17

50 most cited of 126 All 126 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning
  2. 2020MOPO: Model-based Offline Policy Optimization
  3. 2022A Unified Framework for Alternating Offline Model Training and Policy Learning
  4. 2021Offline RL With Resource Constrained Online Deployment
  5. 2023A Survey on Offline Model-Based Reinforcement Learning
  6. 2022Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learning
  7. 2023Finetuning Offline World Models in the Real World
  8. 2025A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory