# Deep Directly-Trained Spiking Neural Networks for Object Detection

arXiv: https://arxiv.org/abs/2307.11411 · HTML: https://mlsanity.com/paper/2307.11411/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Exponential expressivity in deep neural networks through transient chaos](https://mlsanity.com/paper/1606.05340/) (2016)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [A Large Scale Event-based Detection Dataset for Automotive](https://mlsanity.com/paper/2001.08499/) (2020)
- [Attention Spiking Neural Networks](https://mlsanity.com/paper/2209.13929/) (2022)
- [Temporal-wise Attention Spiking Neural Networks for Event Streams Classification](https://mlsanity.com/paper/2107.11711/) (2021)
- [Object Detection with Spiking Neural Networks on Automotive Event Data](https://mlsanity.com/paper/2205.04339/) (2022)
- [Advancing Spiking Neural Networks towards Deep Residual Learning](https://mlsanity.com/paper/2112.08954/) (2021)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Spike Calibration: Fast and Accurate Conversion of Spiking Neural Network for Object Detection and Segmentation](https://mlsanity.com/paper/2207.02702/) (2022)
- [Object Detection in Autonomous Vehicles: Status and Open Challenges](https://mlsanity.com/paper/2201.07706/) (2022)
- [A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks](https://mlsanity.com/paper/2001.00254/) (2020)
- [Spiking SiamFC++: Deep Spiking Neural Network for Object Tracking](https://mlsanity.com/paper/2209.12010/) (2022)
- [Sparse Compressed Spiking Neural Network Accelerator for Object Detection](https://mlsanity.com/paper/2205.00778/) (2022)

## Built on by (most cited first, 30 of 77)

- [Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips](https://mlsanity.com/paper/2404.03663/) (2024)
- [Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training](https://mlsanity.com/paper/2411.16061/) (2024)
- [SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms](https://mlsanity.com/paper/2406.03287/) (2024)
- [Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset](https://mlsanity.com/paper/2412.06647/) (2024)
- [Spiking Vision Transformer with Saccadic Attention](https://mlsanity.com/paper/2502.12677/) (2025)
- [SFOD: Spiking Fusion Object Detector](https://mlsanity.com/paper/2403.15192/) (2024)
- [EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2403.12574/) (2024)
- [Learning A Spiking Neural Network for Efficient Image Deraining](https://mlsanity.com/paper/2405.06277/) (2024)
- [Event-based Tiny Object Detection: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2506.23575/) (2025)
- [Autonomous Driving with Spiking Neural Networks](https://mlsanity.com/paper/2405.19687/) (2024)
- [Quantized Spike-driven Transformer](https://mlsanity.com/paper/2501.13492/) (2025)
- [Rethinking Spiking Neural Networks from an Ensemble Learning Perspective](https://mlsanity.com/paper/2502.14218/) (2025)
- [A Recurrent YOLOv8-based framework for Event-Based Object Detection](https://mlsanity.com/paper/2408.05321/) (2024)
- [Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detection](https://mlsanity.com/paper/2402.01287/) (2024)
- [Adaptive Event Stream Slicing for Open-Vocabulary Event-Based Object Detection via Vision-Language Knowledge Distillation](https://mlsanity.com/paper/2510.00681/) (2025)
- [SU-YOLO: Spiking Neural Network for Efficient Underwater Object Detection](https://mlsanity.com/paper/2503.24389/) (2025)
- [Hybrid Spiking Vision Transformer for Object Detection with Event Cameras](https://mlsanity.com/paper/2505.07715/) (2025)
- [Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention](https://mlsanity.com/paper/2403.10173/) (2024)
- [SDiT: Spiking Diffusion Model with Transformer](https://mlsanity.com/paper/2402.11588/) (2024)
- [Deep Event-based Object Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2405.03995/) (2024)
- [SpikeVideoFormer: An Efficient Spike-Driven Video Transformer with Hamming Attention and O(T) Complexity](https://mlsanity.com/paper/2505.10352/) (2025)
- [When Person Re-Identification Meets Event Camera: A Benchmark Dataset and An Attribute-guided Re-Identification Framework](https://mlsanity.com/paper/2507.13659/) (2025)
- [TP-Spikformer: Token Pruned Spiking Transformer](https://mlsanity.com/paper/2603.00527/) (2026)
- [ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition](https://mlsanity.com/paper/2409.01564/) (2024)
- [Improvement of Spiking Neural Network with Bit Planes and Color Models](https://mlsanity.com/paper/2410.08229/) (2024)
- [SpikeBottleNet: Spike-Driven Feature Compression Architecture for Edge-Cloud Co-Inference](https://mlsanity.com/paper/2410.08673/) (2024)
- [Exploiting Label Skewness for Spiking Neural Networks in Federated Learning](https://mlsanity.com/paper/2412.17305/) (2024)
- [Fully Spiking Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2312.01742/) (2023)
- [Embedded event based object detection with spiking neural network](https://mlsanity.com/paper/2406.17617/) (2024)
- [PowerYOLO: Mixed Precision Model for Hardware Efficient Object Detection with Event Data](https://mlsanity.com/paper/2407.08272/) (2024)

## Abstract (arXiv)

Spiking neural networks (SNNs) are brain-inspired energy-efficient models that encode information in spatiotemporal dynamics. Recently, deep SNNs trained directly have shown great success in achieving high performance on classification tasks with very few time steps. However, how to design a directly-trained SNN for the regression task of object detection still remains a challenging problem. To address this problem, we propose EMS-YOLO, a novel directly-trained SNN framework for object detection, which is the first trial to train a deep SNN with surrogate gradients for object detection rather than ANN-SNN conversion strategies. Specifically, we design a full-spike residual block, EMS-ResNet, which can effectively extend the depth of the directly-trained SNN with low power consumption. Furthermore, we theoretically analyze and prove the EMS-ResNet could avoid gradient vanishing or exploding. The results demonstrate that our approach outperforms the state-of-the-art ANN-SNN conversion methods (at least 500 time steps) in extremely fewer time steps (only 4 time steps). It is shown that our model could achieve comparable performance to the ANN with the same architecture while consuming 5.83 times less energy on the frame-based COCO Dataset and the event-based Gen1 Dataset.