mlsanityData to 7 Oct 2026

Real-world robot offline RL

86 papers1 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: robot, manipulation, reinforcement learning, reinforcement, robotic

Key facts. Real-world robot offline RL: 86 papers on arXiv since 2019 (data through 7 Oct 2026). 9 of its 86 mapped papers (10%) cite “COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning” (2020, arXiv 2010.14500). 7 of its 86 mapped papers (8%) cite “What Matters in Learning from Offline Human Demonstrations for Robot Manipulation” (2021, arXiv 2108.03298). 3 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

3 new since 17 Sep

  1. Teaching a Robot Dog New Tricks: Diverse Quadruped Skills via Combined Reinforcement and Imitation Learning with Adversarial Task Selection
  2. SynIL: Leveraging Synergy for Offline Imitation Learning from Imperfect Demonstration Datasets also fits: Imitation learning methods
  3. Beyond Kinematics: Benchmarking Simulation Fidelity for Muscle-Driven Imitation Learning also fits: Robot manipulation imitation learning

Foundations

Most cited by papers here

  1. COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning2020cited by 9 here
  2. What Matters in Learning from Offline Human Demonstrations for Robot Manipulation2021cited by 7 here
  3. Offline Meta-Reinforcement Learning for Industrial Insertion2021cited by 4 here
  4. Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills2021cited by 3 here
  5. LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion2023cited by 3 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 4 here
  2. 2019Behavior Regularized Offline Reinforcement Learningcited by 8 here
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 8 here
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 15 here
  5. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 5 here
  6. 2020MOPO: Model-based Offline Policy Optimizationcited by 7 here
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 6 here
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 13 here
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 15 here
15 more it builds on, not in this chain
  1. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 3 here
  2. 2018Behavioral Cloning from Observationcited by 4 here
  3. 2018Generative Adversarial Imitation from Observationcited by 3 here
  4. 2018An Algorithmic Perspective on Imitation Learningcited by 6 here
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 20 here
  6. 2020Critic Regularized Regressioncited by 7 here
  7. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 9 here
  8. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 4 here
  9. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 4 here
  10. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 5 here
  11. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 7 here
  12. 2021Implicit Behavioral Cloningcited by 5 here
  13. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 4 here
  14. 2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 4 here
  15. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 4 here

The 24 most cited of 171.

Browse

30 of 86, ranked 6–35

  1. 6Scaling data-driven robotics with reward sketching and batch reinforcement learning20192
  2. 7KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture20251
  3. 8Equivariant Offline Reinforcement Learning20241
  4. 9Learning Speed-Adaptive Walking Agent Using Imitation Learning with Physics-Informed Simulation20241
  5. 10Benchmarking Offline Reinforcement Learning on Real-Robot Hardware20231
  6. 11Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation20231
  7. 12How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation20221
  8. 13Energy-based Legged Robots Terrain Traversability Modeling via Deep Inverse Reinforcement Learning20221
  9. 14Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials20221
  10. 15Model-Based Inverse Reinforcement Learning from Visual Demonstrations20201
All 30
  1. 16RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulation20260
  2. 17AdaptManip: Learning Adaptive Whole-Body Object Lifting and Delivery with Online Recurrent State Estimation20260
  3. 18Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets20260
  4. 19Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation20260
  5. 20Learning Quadruped Walking from Seconds of Demonstration20260
  6. 21Imitation Learning from Human Motion Alone Does Not Guarantee Biomechanically Plausible Gait Kinetics20260
  7. 22Physical Imitation Learning: Distilling Control Policies into Passive Elasticity20260
  8. 23BAT: Balancing Agility and Stability via Online Policy Switching for Long-Horizon Whole-Body Humanoid Control20260
  9. 24MoRI: Mixture of RL and IL Experts for Long-Horizon Manipulation Tasks20260
  10. 25Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers20260
  11. 26E2DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation20260
  12. 27ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting20260
  13. 28Constraint-Aware Diffusion Priors for High-Fidelity and Versatile Quadruped Locomotion20260
  14. 29Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos20260
  15. 30HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning20260
  16. 31Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards20260
  17. 32Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots20260
  18. 33Kine2Go: Kinematic dataset for the Unitree Go2 robot with diverse gaits and motions20260
  19. 34Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL20260
  20. 35RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation20260