mlsanityData to 14 Oct 2026

Learning and Transfer of Modulated Locomotor Controllers

arXiv 1610.05182 ↗First posted 17 Oct 2016cs.RO cs.AIcited by 59

Not on the map. Why?

Abstract (arXiv)

We study a novel architecture and training procedure for locomotion tasks. A high-frequency, low-level "spinal" network with access to proprioceptive sensors learns sensorimotor primitives by training on simple tasks. This pre-trained module is fixed and connected to a low-frequency, high-level "cortical" network, with access to all sensors, which drives behavior by modulating the inputs to the spinal network. Where a monolithic end-to-end architecture fails completely, learning with a pre-trained spinal module succeeds at multiple high-level tasks, and enables the effective exploration required to learn from sparse rewards. We test our proposed architecture on three simulated bodies: a 16-dimensional swimming snake, a 20-dimensional quadruped, and a 54-dimensional humanoid. Our results are illustrated in the accompanying video at https://youtu.be/sboPYvhpraQ

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 14; 2022: 24; 2023: 7; 2024: 5; 2025: 4; 2026: 4.

59 papers.

Sort:

By area:

  1. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  2. 2022Deep Hierarchical Planning from Pixelscited by 101
  3. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  4. 2023Modular Deep Learningcited by 83
  5. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  6. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 54
  7. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  8. 2021Learning to Jump from Pixelscited by 45
  9. 2023Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Modelscited by 37
  10. 2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 34
  11. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  12. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
38 more
  1. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  2. 2022Towards Versatile Embodied Navigationcited by 26
  3. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 24
  4. 2021Hierarchical Reinforcement Learning with Timed Subgoalscited by 21
  5. 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
  6. 2021AMP: Adversarial Motion Priors for Stylized Physics-Based Character Controlcited by 18
  7. 2021Composable Energy Policies for Reactive Motion Generation and Reinforcement Learningcited by 16
  8. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  9. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimizationcited by 12
  10. 2023Hierarchical Reinforcement Learning for Power Network Topology Controlcited by 11
  11. 2024Language-guided Skill Learning with Temporal Variational Inferencecited by 11
  12. 2022Cascaded Compositional Residual Learning for Complex Interactive Behaviorscited by 10
  13. 2025SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blendingcited by 9
  14. 2022Meta-Learning Parameterized Skillscited by 7
  15. 2022ASPiRe:Adaptive Skill Priors for Reinforcement Learningcited by 7
  16. 2021Training Transition Policies via Distribution Matching for Complex Taskscited by 6
  17. 2022ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characterscited by 6
  18. 2022Simple Emergent Action Representations from Multi-Task Policy Trainingcited by 6
  19. 2022SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Explorationcited by 6
  20. 2023Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domainscited by 6
  21. 2021Residual Model Learning for Microrobot Controlcited by 5
  22. 2022Forgetting and Imbalance in Robot Lifelong Learning with Off-policy Datacited by 5
  23. 2022RobustAnalog: Fast Variation-Aware Analog Circuit Design Via Multi-task RLcited by 5
  24. 2025Scalable Option Learning in High-Throughput Environmentscited by 5
  25. 2021Online Baum-Welch algorithm for Hierarchical Imitation Learningcited by 3
  26. 2022Neural Circuit Architectural Priors for Embodied Controlcited by 3
  27. 2022Learning Pneumatic Non-Prehensile Manipulation with a Mobile Blowercited by 2
  28. 2022Learning Design and Construction with Varying-Sized Materials via Prioritized Memory Resetscited by 2
  29. 2023Adaptive trajectory-constrained exploration strategy for deep reinforcement learningcited by 2
  30. 2024Neural Circuit Architectural Priors for Quadruped Locomotioncited by 2
  31. 2021Towards Autonomous Pipeline Inspection with Hierarchical Reinforcement Learningcited by 1
  32. 2023Centralized Cooperative Exploration Policy for Continuous Control Taskscited by 1
  33. 2026Joint Learning of Hierarchical Neural Options and Abstract World Modelcited by 1
  34. 2021Towards autonomous artificial agents with an active self: modeling sense of control in situated actioncited by 0
  35. 2022Boosting Exploration in Multi-Task Reinforcement Learning using Adversarial Networkscited by 0
  36. 2022Matching options to tasks using Option-Indexed Hierarchical Reinforcement Learningcited by 0
  37. 2022What deep reinforcement learning tells us about human motor learning and vice-versacited by 0
  38. 2022Hierarchical Decentralized Deep Reinforcement Learning Architecture for a Simulated Four-Legged Agentcited by 0

50 most cited of 59 All 59 (JSON)