mlsanityData to 7 Oct 2026

Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domains

arXiv 2302.12617 ↗First posted 24 Feb 2023cs.RO cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper we study the problem of learning multi-step dynamics prediction models (jumpy models) from unlabeled experience and their utility for fast inference of (high-level) plans in downstream tasks. In particular we propose to learn a jumpy model alongside a skill embedding space offline, from previously collected experience for which no labels or reward annotations are required. We then investigate several options of harnessing those learned components in combination with model-based planning or model-free reinforcement learning (RL) to speed up learning on downstream tasks. We conduct a set of experiments in the RGB-stacking environment, showing that planning with the learned skills and the associated model can enable zero-shot generalization to new tasks, and can further speed up training of policies via reinforcement learning. These experiments demonstrate that jumpy models which incorporate temporal abstraction can facilitate planning in long-horizon tasks in which standard dynamics models fail.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 43 of its 53 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

43 of 53 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2016Layer Normalizationcited by 5,579
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2018World Modelscited by 1,444
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  10. 2020Mastering Atari with Discrete World Modelscited by 990
  11. 2018Maximum a Posteriori Policy Optimisationcited by 318
  12. 2019Learning Latent Plans from Playcited by 310
31 more
  1. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  2. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  3. 2020Critic Regularized Regressioncited by 269
  4. 2016Variational Intrinsic Controlcited by 262
  5. 2018Graph networks as learnable physics engines for inference and controlcited by 242
  6. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  7. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  8. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  9. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
  10. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  11. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  12. 2018Neural probabilistic motor primitives for humanoid controlcited by 96
  13. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  14. 2019InfoBot: Transfer and Exploration via the Information Bottleneckcited by 78
  15. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  16. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  17. 2013Empowerment -- an Introductioncited by 65
  18. 2016Learning and Transfer of Modulated Locomotor Controllerscited by 59
  19. 2019Information asymmetry in KL-regularized RLcited by 53
  20. 2020Behavior Priors for Efficient Reinforcement Learningcited by 40
  21. 2018Time-Agnostic Prediction: Predicting Predictable Video Framescited by 37
  22. 2020Hierarchical Deep Learning of Multiscale Differential Equation Time-Stepperscited by 37
  23. 2019Dynamics-aware Embeddingscited by 34
  24. 2020Data-efficient Hindsight Off-policy Option Learningcited by 31
  25. 2019Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Modelscited by 26
  26. 2020Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planningcited by 26
  27. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  28. 2021Evaluating model-based planning and planner amortization for continuous controlcited by 15
  29. 2022MO2: Model-Based Offline Optionscited by 10
  30. 2020Local Search for Policy Iteration in Continuous Controlcited by 9
  31. 2021Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 0; 2026: 3.

6 papers.

  1. 2024Simple Hierarchical Planning with Diffusioncited by 71
  2. 2023A Generalist Dynamics Model for Controlcited by 24
  3. 2024Learning Abstract World Model for Value-preserving Planning with Optionscited by 4
  4. 2026Compositional Planning with Jumpy World Modelscited by 4
  5. 2026Offline Reinforcement Learning with Universal Horizon Modelscited by 1
  6. 2026Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Predictioncited by 0