mlsanityData to 14 Oct 2026

Hardware-Software Collaborative Computing of Photonic Spiking Reinforcement Learning for Robotic Continuous Control

arXiv 2512.00427 ↗First posted 29 Nov 2025cs.RO physics.opticscited by 0

On the map: Deep reinforcement learning methods → Spiking and neuromorphic RL

Abstract (arXiv)

Robotic continuous control tasks impose stringent demands on the energy efficiency and latency of computing architectures due to their high-dimensional state spaces and real-time interaction requirements. Conventional electronic computing platforms face computational bottlenecks, whereas the fusion of photonic computing and spiking reinforcement learning (RL) offers a promising alternative. Here, we propose a novel computing architecture based on photonic spiking RL, which integrates the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm with spiking neural network (SNN). The proposed architecture employs an optical-electronic hybrid computing paradigm wherein a silicon photonic Mach-Zehnder interferometer (MZI) chip executes linear matrix computations, while nonlinear spiking activations are performed in the electronic domain. Experimental validation on the Pendulum-v1 and HalfCheetah-v2 benchmarks demonstrates the system capability for software-hardware co-inference, achieving a control policy reward of 5831 on HalfCheetah-v2, a 23.33% reduction in convergence steps, and an action deviation below 2.2%. Notably, this work represents the first application of a programmable MZI photonic computing chip to robotic continuous control tasks, attaining an energy efficiency of 1.39 TOPS/W and an ultralow computational latency of 120 ps. Such performance underscores the promise of photonic spiking RL for real-time decision-making in autonomous and industrial robotic systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spiking and neuromorphic RL, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 80 references that are in the map. First posted on arXiv 29 Nov 2025.

Builds on

31 of 80 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2016OpenAI Gymcited by 1,736
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2018DeepMind Control Suitecited by 956
  8. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  9. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  10. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  11. 2024Reinforcement learningcited by 247
  12. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 152
19 more
  1. 2017Socially Aware Motion Planning with Deep Reinforcement Learningcited by 136
  2. 2016Neuromorphic Silicon Photonic Networkscited by 68
  3. 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 49
  4. 2020Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardwarecited by 36
  5. 2021Average-Reward Off-Policy Policy Evaluation with Function Approximationcited by 31
  6. 2021Human-Level Control through Directly-Trained Deep Spiking Q-Networkscited by 25
  7. 2021Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximationcited by 11
  8. 2020Learning View and Target Invariant Visual Servoing for Navigationcited by 10
  9. 2016Deep Reinforcement Learning for Tensegrity Robot Locomotioncited by 7
  10. 2019Distributed Learning of Decentralized Control Policies for Articulated Mobile Robotscited by 7
  11. 2024Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directionscited by 7
  12. 2020Robotic Arm Control and Task Training through Deep Reinforcement Learningcited by 6
  13. 2023In-memory photonic dot-product engine with electrically programmable weight bankscited by 6
  14. 2023Deep Photonic Reservoir Computer for Speech Recognitioncited by 5
  15. 2017Role Playing Learning for Socially Concomitant Mobile Robot Navigationcited by 4
  16. 2024Neuromorphic Control of a Pendulumcited by 4
  17. 2019Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to ATARI gamescited by 1
  18. 2021A Quadratic Actor Network for Model-Free Reinforcement Learningcited by 1
  19. 2022Enabling Deep Reinforcement Learning on Energy Constrained Devices at the Edge of the Networkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning
  2. 2025Hardware-Aware Fine-Tuning of Spiking Q-Networks on the SpiNNaker2 Neuromorphic Platform
  3. 2024Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment
  4. 2025Fully Spiking Actor-Critic Neural Network for Robotic Manipulation
  5. 2025SpikeRL: A Scalable and Energy-efficient Framework for Deep Spiking Reinforcement Learning
  6. 2026Otters++: A Time-to-first-spike Based Energy Efficient Optical Spiking Transformer
  7. 2024Enabling Energy-Efficient Simultaneous Multi-Task Reinforcement Learning through Spiking Neural Networks with Active Dendrites for Bio-inspired Generalist Agents
  8. 2025Population-Coded Spiking Neural Networks for High-Dimensional Robotic Control