EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning
On the map: World models → Latent predictive world models also fits: Latent dynamics models and model-based RL
Abstract (arXiv)
A robot should be able to learn through experiments how unfamiliar objects behave and interact, then plan with that knowledge. It need not start from scratch: physics engines supply knowledge of motion and contact, but can omit entire mechanisms, such as glue curing, water heating, or wind. We present EMPIRIC, an agent that learns a residual world model: a physics engine extended with code for the missing mechanisms. The learned programs can introduce new forces, constraints, and hidden state, and Bayesian inference estimates their parameters and states from noisy observations. The resulting model lets the agent predict the outcomes of actions, choose informative experiments, and revise its hypotheses when predictions fail. Across five simulated domains, EMPIRIC learns interpretable, reusable models, and solves more tasks with fewer environment interactions than all three baselines. On a physical robot, it learns wind forces and domino masses to solve a manipulation task. Website and code: https://yichao-liang.github.io/empiric
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 38 of its 94 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
38 of 94 references.
Closest ancestors · 1
Same direction, newest first
Other references · 37
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2020Planning to Explore via Self-Supervised World Modelscited by 368
- 2020Integrated Task and Motion Planningcited by 290
- 2013A General Framework for Updating Belief Distributionscited by 175
- 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
- 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
- 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 111
- 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
- 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 88
25 more
- 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 85
- 2020NeuralSim: Augmenting Differentiable Simulators with Neural Networkscited by 82
- 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 69
- 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 69
- 2026ASPIRE: Agentic /Skills Discovery for Roboticscited by 53
- 2021Learning Neuro-Symbolic Relational Transition Models for Bilevel Planningcited by 43
- 2025Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulationcited by 43
- 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 40
- 2024VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planningcited by 39
- 2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real Worldcited by 38
- 2026ARC-AGI-3: A New Challenge for Frontier Agentic Intelligencecited by 37
- 2025Scalable Real2Sim: Physics-Aware Asset Generation Via Robotic Pick-and-Place Setupscited by 30
- 2024ASID: Active Exploration for System Identification in Robotic Manipulationcited by 28
- 2025PoE-World: Compositional World Modeling with Products of Programmatic Expertscited by 26
- 2025Code World Models for General Game Playingcited by 26
- 2019Combining Physical Simulators and Object-Based Networks for Controlcited by 18
- 2026SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluationcited by 16
- 2025Synthesizing world models for bilevel planningcited by 14
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 14
- 2025LLM-Guided Probabilistic Program Induction for POMDP Model Estimationcited by 12
- 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agentscited by 8
- 2022Task-Directed Exploration in Continuous POMDPs for Robotic Manipulation of Articulated Objectscited by 7
- 2024One-Shot Real-to-Sim via End-to-End Differentiable Simulation and Renderingcited by 6
- 2026Coding Agents for Generalized Task and Motion Planning Problemscited by 2
- 2026Learning POMDP World Models from Observations with Language-Model Priorscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation
- 2026Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning
- 2026World Models for Robotic Manipulation: A Survey
- 2026World Model for Robot Learning: A Comprehensive Survey
- 2026BWM: A Low-Cost High-Fidelity World Simulator for Robot Learning
- 2026Copper-Policy: Focus on the Representation for Robust Robot Manipulation
- 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination
- 2026Nautilus: From One Prompt to Plug-and-Play Robot Learning