mlsanityData to 7 Oct 2026

Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents

arXiv 2212.09033 ↗First posted 18 Dec 2022cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In reinforcement learning applications like robotics, agents usually need to deal with various input/output features when specified with different state/action spaces by their developers or physical restrictions. This indicates unnecessary re-training from scratch and considerable sample inefficiency, especially when agents follow similar solution steps to achieve tasks. In this paper, we aim to transfer similar high-level goal-transition knowledge to alleviate the challenge. Specifically, we propose PILoT, i.e., Planning Immediate Landmarks of Targets. PILoT utilizes the universal decoupled policy optimization to learn a goal-conditioned state planner; then, distills a goal-planner to plan immediate landmarks in a model-free style that can be shared among different agents. In our experiments, we show the power of PILoT on various transferring challenges, including few-shot transferring across action spaces and dynamics, from low-dimensional vector states to image inputs, from simple robot to complicated morphology; and we also illustrate a zero-shot transfer solution from a simple 2D navigation task to the harder Ant-Maze task.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 18 Dec 2022.

Builds on

33 of 39 references.

References · 33

  1. 2017Hindsight Experience Replaycited by 1,096
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  3. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  4. 2016The Option-Critic Architecturecited by 433
  5. 2016Learning to reinforcement learncited by 369
  6. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  7. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  8. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  9. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  10. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
  11. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  12. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
21 more
  1. 2018Unity: A General Platform for Intelligent Agentscited by 232
  2. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  3. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  4. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  5. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
  6. 2018Universal Successor Features Approximatorscited by 102
  7. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  8. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 84
  9. 2022MetaMorph: Learning Universal Controllers with Transformerscited by 78
  10. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  11. 2020My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Controlcited by 68
  12. 2016Learning and Transfer of Modulated Locomotor Controllerscited by 59
  13. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  14. 2019Sub-policy Adaptation for Hierarchical Reinforcement Learningcited by 47
  15. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  16. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  17. 2022Transfer RL across Observation Feature Spaces via Model-Based Regularizationcited by 18
  18. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  19. 2021MapGo: Model-Assisted Policy Optimization for Goal-Oriented Taskscited by 13
  20. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimizationcited by 12
  21. 2017Sensor Fusion for Robot Control through Deep Reinforcement Learningcited by 7

Built on by

Not cited yet.