mlsanityData to 14 Oct 2026

Autonomous Driving with Spiking Neural Networks

arXiv 2405.19687 ↗First posted 30 May 2024cs.NE cs.CVcited by 13

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

Autonomous driving demands an integrated approach that encompasses perception, prediction, and planning, all while operating under strict energy constraints to enhance scalability and environmental sustainability. We present Spiking Autonomous Driving (SAD), the first unified Spiking Neural Network (SNN) to address the energy challenges faced by autonomous driving systems through its event-driven and energy-efficient nature. SAD is trained end-to-end and consists of three main modules: perception, which processes inputs from multi-view cameras to construct a spatiotemporal bird's eye view; prediction, which utilizes a novel dual-pathway with spiking neurons to forecast future states; and planning, which generates safe trajectories considering predicted occupancy, traffic rules, and ride comfort. Evaluated on the nuScenes dataset, SAD achieves competitive performance in perception, prediction, and planning tasks, while drawing upon the energy efficiency of SNNs. This work highlights the potential of neuromorphic computing to be applied to energy-efficient autonomous driving, a critical step toward sustainable and safety-critical automotive technology. Our code is available at urlhttps://github.com/ridgerchu/SAD.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 67 of its 87 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

67 of 87 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023End-to-end Autonomous Driving: Challenges and Frontiers29 Jun
  2. 2023Hidden Biases of End-to-End Driving Models13 Jun
  3. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May
  4. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning15 Jul
  5. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline16 Jun
  6. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving31 May
  7. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr
  8. 2021Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation18 Apr
  9. 2021End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving18 Apr
  10. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 57

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  3. 2016End to End Learning for Self-Driving Carscited by 1,009
  4. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  5. 2019Learning by Cheatingcited by 383
  6. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  7. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  8. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  9. 2018Learning to Drive in a Daycited by 273
  10. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 271
  11. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 259
  12. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
38 more
  1. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
  2. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  3. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 177
  4. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  5. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 174
  6. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  7. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 169
  8. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 153
  9. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  10. 2023Spike-driven Transformercited by 146
  11. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 144
  12. 2020Video Panoptic Segmentationcited by 142
  13. 2021Learning to drive from a world on railscited by 114
  14. 2022Attention Spiking Neural Networkscited by 107
  15. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 103
  16. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 97
  17. 2023SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networkscited by 88
  18. 2019Urban Driving with Conditional Imitation Learningcited by 86
  19. 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
  20. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 84
  21. 2021Advancing Spiking Neural Networks towards Deep Residual Learningcited by 79
  22. 2023Deep Directly-Trained Spiking Neural Networks for Object Detectioncited by 77
  23. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  24. 2021Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentationcited by 55
  25. 2023SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiationcited by 48
  26. 2020Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networkscited by 47
  27. 2022Masked Spiking Transformercited by 39
  28. 2022TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networkscited by 35
  29. 2020Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applicationscited by 34
  30. 2021Event-driven Vision and Control for UAVs on a Neuromorphic Chipcited by 34
  31. 2023Brain-inspired learning in artificial neural networks: a reviewcited by 26
  32. 2023Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networkscited by 26
  33. 2021Fully Spiking Variational Autoencodercited by 22
  34. 2023Coaching a Teachable Studentcited by 19
  35. 2023Spiking Denoising Diffusion Probabilistic Modelscited by 19
  36. 2020Long Short-Term Memory Spiking Networks and Their Applicationscited by 16
  37. 2019MorphIC: A 65-nm 738k-Synapse/mm2 Quad-Core Binary-Weight Digital Neuromorphic Processor with Stochastic Spike-Driven Online Learningcited by 13
  38. 2022Spiking neural networks for nonlinear regressioncited by 10

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 6.

13 papers.

Sort:

By area:

  1. 2026SpikeTrack: A Spike-driven Framework for Efficient Visual Trackingcited by 5
  2. 2024Flashy Backdoor: Real-world Environment Backdoor Attack on SNNs with DVS Camerascited by 3
  3. 2025Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Backcited by 3
  4. 2025SpikingRTNH: Spiking Neural Network for 4D Radar Object Detectioncited by 2
  5. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  6. 2025On the Importance of Neural Membrane Potential Leakage for LIDAR-based Robot Obstacle Avoidance using Spiking Neural Networkscited by 0
  7. 2025Spiking Neural Networks for Continuous Control via End-to-End Model-Based Learningcited by 0
  8. 2025A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networkscited by 0
  9. 2026SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecastingcited by 0
  10. 2026Neuromorphic LiDAR-based Bird's Eye View Object Detection using Energy-efficient Spiking Neural Networkscited by 0
  11. 2026Exploring Solver-Level Warmstarting for Neural Network Verificationcited by 0
  12. 2026SpikeLite: Lightweight Spiking Neural Networks for Time-Series Forecastingcited by 0
1 more
  1. 2026SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Expertscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Neuromorphic Computing for Embodied Intelligence in Autonomous Systems: Current Trends, Challenges, and Future Directions
  2. 2024SNN4Agents: A Framework for Developing Energy-Efficient Embodied Spiking Neural Networks for Autonomous Agents
  3. 2026SDPAD: A Fully Spike-Driven Pipeline for End-to-End Autonomous Driving
  4. 2025Energy-Efficient Autonomous Driving with Adaptive Perception and Robust Decision
  5. 2026Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Driving
  6. 2025New Spiking Architecture for Multi-Modal Decision-Making in Autonomous Vehicles
  7. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey
  8. 2024Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detection