# Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardware

arXiv: https://arxiv.org/abs/2003.01157 · HTML: https://mlsanity.com/paper/2003.01157/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2020.

## Built on by (most cited first, 30 of 36)

- [Representation Learning for Event-based Visuomotor Policies](https://mlsanity.com/paper/2103.00806/) (2021)
- [Deep Reinforcement Learning with Spiking Q-learning](https://mlsanity.com/paper/2201.09754/) (2022)
- [Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity](https://mlsanity.com/paper/2111.01760/) (2021)
- [Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning](https://mlsanity.com/paper/2401.05444/) (2024)
- [Fully Spiking Neural Network for Legged Robots](https://mlsanity.com/paper/2310.05022/) (2023)
- [Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control](https://mlsanity.com/paper/2505.24161/) (2025)
- [Distilling Neuron Spike with High Temperature in Reinforcement Learning Agents](https://mlsanity.com/paper/2108.10078/) (2021)
- [A Low Latency Adaptive Coding Spiking Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2211.11760/) (2022)
- [Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network](https://mlsanity.com/paper/2301.10292/) (2022)
- [Mixture of personality improved Spiking actor network for efficient multi-agent cooperation](https://mlsanity.com/paper/2305.05898/) (2023)
- [Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learning](https://mlsanity.com/paper/2301.07275/) (2023)
- [Noisy Spiking Actor Network for Exploration](https://mlsanity.com/paper/2403.04162/) (2024)
- [GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL](https://mlsanity.com/paper/2404.15597/) (2024)
- [Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons](https://mlsanity.com/paper/2506.03392/) (2025)
- [CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning](https://mlsanity.com/paper/2509.23791/) (2025)
- [Biologically-Plausible Topology Improved Spiking Actor Network for Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2403.20163/) (2024)
- [Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning](https://mlsanity.com/paper/2106.07854/) (2021)
- [Including STDP to eligibility propagation in multi-layer recurrent spiking neural networks](https://mlsanity.com/paper/2201.07602/) (2022)
- [Spiking Neural Network Feature Discrimination Boosts Modality Fusion](https://mlsanity.com/paper/2502.10423/) (2025)
- [Population-Coded Spiking Neural Networks for High-Dimensional Robotic Control](https://mlsanity.com/paper/2510.10516/) (2025)
- [Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks](https://mlsanity.com/paper/2510.24461/) (2025)
- [Dynamic Weight Adaptation in Spiking Neural Networks Inspired by Biological Homeostasis](https://mlsanity.com/paper/2511.17563/) (2025)
- [Error Amplification Limits ANN-to-SNN Conversion in Continuous Control](https://mlsanity.com/paper/2601.21778/) (2026)
- [A Novel Neuromorphic Processors Realization of Spiking Deep Reinforcement Learning for Portfolio Management](https://mlsanity.com/paper/2203.14159/) (2022)
- [EEGSN: Towards Efficient Low-latency Decoding of EEG with Graph Spiking Neural Networks](https://mlsanity.com/paper/2304.07655/) (2023)
- [SpikeRL: A Scalable and Energy-efficient Framework for Deep Spiking Reinforcement Learning](https://mlsanity.com/paper/2502.17496/) (2025)
- [AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services](https://mlsanity.com/paper/2503.04418/) (2025)
- [Energy-Efficient Deep Reinforcement Learning with Spiking Transformers](https://mlsanity.com/paper/2505.14533/) (2025)
- [Hardware-Aware Fine-Tuning of Spiking Q-Networks on the SpiNNaker2 Neuromorphic Platform](https://mlsanity.com/paper/2507.23562/) (2025)
- [Fully Spiking Actor-Critic Neural Network for Robotic Manipulation](https://mlsanity.com/paper/2508.12038/) (2025)

## Abstract (arXiv)

Energy-efficient mapless navigation is crucial for mobile robots as they explore unknown environments with limited on-board resources. Although the recent deep reinforcement learning (DRL) approaches have been successfully applied to navigation, their high energy consumption limits their use in several robotic applications. Here, we propose a neuromorphic approach that combines the energy-efficiency of spiking neural networks with the optimality of DRL and benchmark it in learning control policies for mapless navigation. Our hybrid framework, spiking deep deterministic policy gradient (SDDPG), consists of a spiking actor network (SAN) and a deep critic network, where the two networks were trained jointly using gradient descent. The co-learning enabled synergistic information exchange between the two networks, allowing them to overcome each other's limitations through a shared representation learning. To evaluate our approach, we deployed the trained SAN on Intel's Loihi neuromorphic processor. When validated on simulated and real-world complex environments, our method on Loihi consumed 75 times less energy per inference as compared to DDPG on Jetson TX2, and also exhibited a higher rate of successful navigation to the goal, which ranged from 1% to 4.2% and depended on the forward-propagation timestep size. These results reinforce our ongoing efforts to design brain-inspired algorithms for controlling autonomous robots with neuromorphic hardware.