Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network
Not on the map. Why?
Abstract (arXiv)
Learning from interaction is the primary way that biological agents acquire knowledge about their environment and themselves. Modern deep reinforcement learning (DRL) explores a computational approach to learning from interaction and has made significant progress in solving various tasks. However, despite its power, DRL still falls short of biological agents in terms of energy efficiency. Although the underlying mechanisms are not fully understood, we believe that the integration of spiking communication between neurons and biologically-plausible synaptic plasticity plays a prominent role in achieving greater energy efficiency. Following this biological intuition, we optimized a spiking policy network (SPN) using a genetic algorithm as an energy-efficient alternative to DRL. Our SPN mimics the sensorimotor neuron pathway of insects and communicates through event-based spikes. Inspired by biological research showing that the brain forms memories by creating new synaptic connections and rewiring these connections based on new experiences, we tuned the synaptic connections instead of weights in the SPN to solve given tasks. Experimental results on several robotic control tasks demonstrate that our method can achieve the same level of performance as mainstream DRL methods while exhibiting significantly higher energy efficiency.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 75 references that are in the map. First posted on arXiv 29 Dec 2022.
Builds on
20 of 75 references.
References · 20
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2015Trust Region Policy Optimizationcited by 2,503
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
- 2016OpenAI Gymcited by 1,684
- 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
- 2017Deep Reinforcement Learning that Matterscited by 727
- 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
- 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
8 more
- 2019Weight Agnostic Neural Networkscited by 71
- 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 44
- 2020Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardwarecited by 32
- 2021Human-Level Control through Directly-Trained Deep Spiking Q-Networkscited by 23
- 2020Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networkscited by 20
- 2022Deep Reinforcement Learning with Spiking Q-learningcited by 15
- 2022Recent Advances and New Frontiers in Spiking Neural Networkscited by 10
- 2019On-Policy Trust Region Policy Optimisation with Replay Bufferscited by 1
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.