# Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention

arXiv: https://arxiv.org/abs/2403.10173 · HTML: https://mlsanity.com/paper/2403.10173/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 51 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [A Large Scale Event-based Detection Dataset for Automotive](https://mlsanity.com/paper/2001.08499/) (2020)
- [Event-based Asynchronous Sparse Convolutional Networks](https://mlsanity.com/paper/2003.09148/) (2020)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Object Detection with Spiking Neural Networks on Automotive Event Data](https://mlsanity.com/paper/2205.04339/) (2022)
- [Deep Directly-Trained Spiking Neural Networks for Object Detection](https://mlsanity.com/paper/2307.11411/) (2023)
- [State Space Models for Event Cameras](https://mlsanity.com/paper/2402.15584/) (2024)
- [LEOD: Label-Efficient Object Detection for Event Cameras](https://mlsanity.com/paper/2311.17286/) (2023)

## Built on by (most cited first, 6 of 6)

- [GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking](https://mlsanity.com/paper/2607.15699/) (2026)
- [Robust PnP on a Neuromorphic Processor for Object Pose Estimation](https://mlsanity.com/paper/2607.16834/) (2026)
- [Neuromorphic Object Detection: An In-Depth Study and Future Directions](https://mlsanity.com/paper/2607.23576/) (2026)
- [BLADE: ReliaBle Dynamic Hardware-Aware SNN-ANN Boundary SeLection for Event-BAseD Object DEtection](https://mlsanity.com/paper/2609.17562/) (2026)
- [Vmem-varphi: Low-Compute Out-of-Distribution Detection in Spiking Neural Networks from Membrane-Potential Statistics](https://mlsanity.com/paper/2610.00350/) (2026)
- [Event-guided Neural Video Compression](https://mlsanity.com/paper/2610.02265/) (2026)

## Abstract (arXiv)

Event cameras offer high temporal resolution and dynamic range with minimal motion blur, making them promising for robust object detection. While Spiking Neural Networks (SNNs) on neuromorphic hardware are often considered for energy-efficient and low latency event-based data processing, they often fall short of Artificial Neural Networks (ANNs) in accuracy and flexibility. Here, we introduce Attention-based Hybrid SNN-ANN backbones for event-based object detection to leverage the strengths of both SNN and ANN architectures. A novel Attention-based SNN-ANN bridge module captures sparse spatial and temporal relations from the SNN layer and converts them into dense feature maps for the ANN part of the backbone. Additionally, we present a variant that integrates DWConvL-STMs to the ANN blocks to capture slower dynamics. This multi-timescale network combines fast SNN processing for short timesteps with long-term dense RNN processing, effectively capturing both fast and slow dynamics. Experimental results demonstrate that our proposed method surpasses SNN-based approaches by significant margins, with results comparable to existing ANN and RNN-based methods. Unlike ANN-only networks, the hybrid setup allows us to implement the SNN blocks on digital neuromorphic hardware to investigate the feasibility of our approach. Extensive ablation studies and implementation on neuromorphic hardware confirm the effectiveness of our proposed modules and architectural choices. Our hybrid SNN-ANN architectures pave the way for ANN-like performance at a drastically reduced parameter, latency, and power budget.