# Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation

arXiv: https://arxiv.org/abs/2003.10026 · HTML: https://mlsanity.com/paper/2003.10026/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2020.

## Built on by (most cited first, 6 of 6)

- [Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization](https://mlsanity.com/paper/2206.03654/) (2022)
- [Neuromorphic adaptive spiking CPG towards bio-inspired locomotion of legged robots](https://mlsanity.com/paper/2101.09709/) (2021)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04511/) (2021)
- [An Interpretable Neural Control Network with Adaptable Online Learning for Sample Efficient Robot Locomotion Learning](https://mlsanity.com/paper/2501.10698/) (2025)
- [Learning Natural and Robust Hexapod Locomotion over Complex Terrains via Motion Priors based on Deep Reinforcement Learning](https://mlsanity.com/paper/2511.03167/) (2025)

## Abstract (arXiv)

Learning to walk -- i.e., learning locomotion under performance and energy constraints continues to be a challenge in legged robotics. Methods such as stochastic gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. These techniques are computationally intensive and often prohibitive for edge applications. These methods rely on complex sensors and pre-processing of data, which further increases energy and latency. Recent advances in spiking neural networks (SNNs) promise a significant reduction in computing owing to the sparse firing of neuros and has been shown to integrate reinforcement learning mechanisms with biologically observed spike time dependent plasticity (STDP). However, training a legged robot to walk by learning the synchronization patterns of central pattern generators (CPG) in an SNN framework has not been shown. This can marry the efficiency of SNNs with synchronized locomotion of CPG based systems providing breakthrough end-to-end learning in mobile robotics. In this paper, we propose a reinforcement based stochastic weight update technique for training a spiking CPG. The whole system is implemented on a lightweight raspberry pi platform with integrated sensors, thus opening up exciting new possibilities.