mlsanityData to 7 Oct 2026

Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation

arXiv 2602.24121 ↗First posted 27 Feb 2026cs.ROcited by 0

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

Current methods in robot learning are fundamentally bottlenecked by one or more of: hand-designed rewards, simulation modeling, or action supervision (e.g. teleoperation) each requiring significant domain expertise, engineering effort, and robot-operator labor. Towards eliminating these bottlenecks, this work pursues observational learning via Inverse Reinforcement Learning from Observation (IRLfO) in which only access to task observations (e.g. video) is assumed. Due to the challenging setting and limitations of RL methods, IRLfO has thus far remained impractical for real-world robot learning. Here, we present the first IRL method to learn visual manipulation in the real world from scratch, and the first real-world demonstration of positive online transfer across visual manipulation tasks from scratch. In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82% success, where RL and BC with equal interaction and demonstration budgets reach only 0% and 12% despite their reward and action supervision. Interactive project page with training videos: https://uwrobotlearning.github.io/mpail2/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 23 of its 57 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

23 of 57 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  5. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  6. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  7. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  8. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  9. 2018Generative Adversarial Imitation from Observationcited by 183
  10. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  11. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  12. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
11 more
  1. 2018Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLcited by 80
  2. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 35
  3. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Controlcited by 35
  4. 2025Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuningcited by 18
  5. 2023Massively Scalable Inverse Reinforcement Learning in Google Mapscited by 11
  6. 2025PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policiescited by 11
  7. 2025Learning Massively Multitask World Models for Continuous Controlcited by 11
  8. 2019Sample-efficient Adversarial Imitation Learning from Observationcited by 6
  9. 2025Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Languagecited by 4
  10. 2025Unifying Model Predictive Path Integral Control, Reinforcement Learning, and Diffusion Models for Optimal Control and Planningcited by 3
  11. 2025Model Predictive Adversarial Imitation Learning for Planning from Observationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrations
  2. 2024Visual IRL for Human-Like Robotic Manipulation
  3. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation
  4. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation
  5. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation
  6. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
  7. 2022How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
  8. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets