Data to 14 Oct 2026

Policies Modulating Trajectory Generators

arXiv 1910.02812 ↗First posted 7 Oct 2019cs.RO cs.AI cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

We propose an architecture for learning complex controllable behaviors by having simple Policies Modulate Trajectory Generators (PMTG), a powerful combination that can provide both memory and prior knowledge to the controller. The result is a flexible architecture that is applicable to a class of problems with periodic motion for which one has an insight into the class of trajectories that might lead to a desired behavior. We illustrate the basics of our architecture using a synthetic control problem, then go on to learn speed-controlled locomotion for a quadrupedal robot by using Deep Reinforcement Learning and Evolutionary Strategies. We demonstrate that a simple linear policy, when paired with a parametric Trajectory Generator for quadrupedal gaits, can induce walking behaviors with controllable speed from 4-dimensional IMU observations alone, and can be learned in under 1000 rollouts. We also transfer these policies to a real robot and show locomotion with controllable forward velocity.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 13; 2023: 12; 2024: 3; 2025: 5; 2026: 3.

49 papers.

Sort:

By area:

2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 89
2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 79
2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 54
2021Learning to Jump from Pixelscited by 45
2023Neural Volumetric Memory for Visual Locomotion Controlcited by 44
2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 41
2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 41
2022CPG-RL: Learning Central Pattern Generators for Quadruped Locomotioncited by 34

37 more

2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 33
2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 30
2023RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotioncited by 26
2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 24
2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 23
2022ViNL: Visual Navigation and Locomotion Over Obstaclescited by 23
2021Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learningcited by 22
2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 20
2023ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robotscited by 19
2021Imitation Learning from MPC for Quadrupedal Multi-Gait Controlcited by 18
2024FLD: Fourier Latent Dynamics for Structured Motion Representation and Learningcited by 18
2021Discovering Diverse Athletic Jumping Strategiescited by 16
2021Reinforcement Learning with Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotioncited by 14
2022Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotioncited by 14
2023Continuous Versatile Jumping Using Learned Action Residualscited by 8
2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
2023Puppeteer and Marionette: Learning Anticipatory Quadrupedal Locomotion Based on Interactions of a Central Pattern Generator and Supraspinal Drivecited by 5
2024ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimizationcited by 4
2021Unlocking Pixels for Reinforcement Learning via Implicit Attentioncited by 3
2022Neural Circuit Architectural Priors for Embodied Controlcited by 3
2022Efficient Learning of Locomotion Skills through the Discovery of Diverse Environmental Trajectory Generator Priorscited by 2
2024Neural Circuit Architectural Priors for Quadruped Locomotioncited by 2
2021PM-FSM: Policies Modulating Finite State Machine for Robust Quadrupedal Locomotioncited by 1
2022Learning Solution Manifolds for Control Problems via Energy Minimizationcited by 1
2022Learning to Exploit Elastic Actuators for Quadruped Locomotioncited by 1
2023Expanding Versatility of Agile Locomotion through Policy Transitions Using Latent State Representationcited by 1
2023MTAC: Hierarchical Reinforcement Learning-based Multi-gait Terrain-adaptive Quadruped Controllercited by 1
2025Ground contact and reaction force sensing for linear policy control of quadruped robotcited by 1
2025ES-Parkour: Advanced Robot Parkour with Bio-inspired Event Camera and Spiking Neural Networkcited by 1
2025Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robotscited by 1
2025Hierarchical Reinforcement Learning and Value Optimization for Challenging Quadruped Locomotioncited by 1
2026Agile perceptive multi-skill locomotion for quadrupedal robots in the wildcited by 1
2022Learning Modular Robot Locomotion from Demonstrationscited by 0
2023Unlock Reliable Skill Inference for Quadruped Adaptive Behavior by Skill Graphcited by 0
2025Energy-Efficient Omnidirectional Locomotion for Wheeled Quadrupeds via Predictive Energy-Aware Nominal Gait Selectioncited by 0
2026ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiderscited by 0
2026GLAMDRING: Gait Learning And Morphology co-Design via Reinforcement LearnING of CPGscited by 0