Back-stepping Experience Replay with Application to Model-free Reinforcement Learning for a Soft Snake Robot
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose a novel technique, Back-stepping Experience Replay (BER), that is compatible with arbitrary off-policy reinforcement learning (RL) algorithms. BER aims to enhance learning efficiency in systems with approximate reversibility, reducing the need for complex reward shaping. The method constructs reversed trajectories using back-stepping transitions to reach random or fixed targets. Interpretable as a bi-directional approach, BER addresses inaccuracies in back-stepping transitions through a distillation of the replay experience during learning. Given the intricate nature of soft robots and their complex interactions with environments, we present an application of BER in a model-free RL approach for the locomotion and navigation of a soft snake robot, which is capable of serpentine motion enabled by anisotropic friction between the body and ground. In addition, a dynamic simulator is developed to assess the effectiveness and efficiency of the BER algorithm, in which the robot demonstrates successful learning (reaching a 100% success rate) and adeptly reaches random targets, achieving an average speed 48% faster than that of the best baseline approach.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 26 references that are in the map. First posted on arXiv 21 Jan 2024.
Builds on
12 of 26 references.
References · 12
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2017Hindsight Experience Replaycited by 1,096
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
- 2017Count-Based Exploration with Neural Density Modelscited by 255
- 2019Goal-conditioned Imitation Learningcited by 159
- 2016Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrationcited by 35
- 2018DORA The Explorer: Directed Outreaching Reinforcement Action-Selectioncited by 27
- 2023Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospectscited by 15
- 2019Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Modelcited by 14
- 2022Reinforcement Learning of CPG-regulated Locomotion Controller for a Soft Snake Robotcited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.