mlsanityData to 7 Oct 2026

Goal-conditioned Batch Reinforcement Learning for Rotation Invariant Locomotion

arXiv 2004.08356 ↗First posted 17 Apr 2020cs.LG cs.AI stat.MLcited by 4

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

We propose a novel approach to learn goal-conditioned policies for locomotion in a batch RL setting. The batch data is collected by a policy that is not goal-conditioned. For the locomotion task, this translates to data collection using a policy learnt by the agent for walking straight in one direction, and using that data to learn a goal-conditioned policy that enables the agent to walk in any direction. The data collection policy used should be invariant to the direction the agent is facing i.e. regardless of its initial orientation, the agent should take the same actions to walk forward. We exploit this property to learn a goal-conditioned policy using two key ideas: (1) augmenting data by generating trajectories with the same actions in different directions, and (2) learning an encoder that enforces invariance between these rotated trajectories with a Siamese framework. We show that our approach outperforms existing RL algorithms on 3-D locomotion agents like Ant, Humanoid and Minitaur.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 23 references that are in the map. First posted on arXiv 17 Apr 2020.

Builds on

15 of 23 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2016OpenAI Gymcited by 1,684
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  9. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  10. 2019Batch Policy Learning under Constraintscited by 242
  11. 2019Goal-conditioned Imitation Learningcited by 159
  12. 2019Planning with Goal-Conditioned Policiescited by 144
3 more
  1. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  2. 2018Learning Actionable Representations with Goal-Conditioned Policiescited by 53
  3. 2019Augmenting learning using symmetry in a biologically-inspired domaincited by 4

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 0; 2023: 1; 2024: 0; 2025: 1.

4 papers.

  1. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  2. 2021Multi-Agent MDP Homomorphic Networkscited by 24
  3. 2023An Investigation of Time Reversal Symmetry in Reinforcement Learningcited by 1
  4. 2025Symmetries-enhanced Multi-Agent Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policies
  2. 2020Learning to Walk in the Real World with Minimal Human Effort
  3. 2026MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins
  4. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning
  5. 2025Flattening Hierarchies with Policy Bootstrapping
  6. 2024Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression
  7. 2023Goal-Conditioned Predictive Coding for Offline Reinforcement Learning
  8. 2023Backward Learning for Goal-Conditioned Policies