# Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision

arXiv: https://arxiv.org/abs/2112.03423 · HTML: https://mlsanity.com/paper/2112.03423/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 59 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Training Deep Spiking Neural Networks using Backpropagation](https://mlsanity.com/paper/1608.08782/) (2016)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks](https://mlsanity.com/paper/2008.03658/) (2020)
- [Surrogate gradients for analog neuromorphic computing](https://mlsanity.com/paper/2006.07239/) (2020)
- [STDP-based spiking deep convolutional neural networks for object recognition](https://mlsanity.com/paper/1611.01421/) (2016)
- [Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain](https://mlsanity.com/paper/1805.08932/) (2018)
- [Stable and expressive recurrent vision models](https://mlsanity.com/paper/2005.11362/) (2020)
- [Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate](https://mlsanity.com/paper/1912.12980/) (2019)
- [Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities](https://mlsanity.com/paper/1811.07802/) (2018)
- [Neuromorphic Visual Odometry System for Intelligent Vehicle Application with Bio-inspired Vision Sensor](https://mlsanity.com/paper/1909.02490/) (2019)

## Built on by (most cited first, 22 of 22)

- [Object Detection with Spiking Neural Networks on Automotive Event Data](https://mlsanity.com/paper/2205.04339/) (2022)
- [SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition](https://mlsanity.com/paper/2308.04369/) (2023)
- [SFOD: Spiking Fusion Object Detector](https://mlsanity.com/paper/2403.15192/) (2024)
- [EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2403.12574/) (2024)
- [A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception](https://mlsanity.com/paper/2303.14176/) (2023)
- [A Recurrent YOLOv8-based framework for Event-Based Object Detection](https://mlsanity.com/paper/2408.05321/) (2024)
- [Adaptive Event Stream Slicing for Open-Vocabulary Event-Based Object Detection via Vision-Language Knowledge Distillation](https://mlsanity.com/paper/2510.00681/) (2025)
- [Spiking-Diffusion: Vector Quantized Discrete Diffusion Model with Spiking Neural Networks](https://mlsanity.com/paper/2308.10187/) (2023)
- [Spiking NeRF: Representing the Real-World Geometry by a Discontinuous Representation](https://mlsanity.com/paper/2311.09077/) (2023)
- [Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention](https://mlsanity.com/paper/2403.10173/) (2024)
- [Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approach](https://mlsanity.com/paper/2507.03765/) (2025)
- [PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining](https://mlsanity.com/paper/2505.05307/) (2025)
- [Deep Event-based Object Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2405.03995/) (2024)
- [Embedded event based object detection with spiking neural network](https://mlsanity.com/paper/2406.17617/) (2024)
- [Towards Efficient Deployment of Hybrid SNNs on Neuromorphic and Edge AI Hardware](https://mlsanity.com/paper/2407.08704/) (2024)
- [EventTracer: Fast Path Tracing-based Event Stream Rendering](https://mlsanity.com/paper/2508.18071/) (2025)
- [Hybrid Layer-Wise ANN-SNN With Surrogate Spike Encoding-Decoding Structure](https://mlsanity.com/paper/2509.24411/) (2025)
- [SGED: A Benchmark dataset for Performance Evaluation of Spiking Gesture Emotion Recognition](https://mlsanity.com/paper/2304.14714/) (2023)
- [Energy-Efficient Visual Search by Eye Movement and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2310.06578/) (2023)
- [Energy-Efficient Eimeria Parasite Detection Using a Two-Stage Spiking Neural Network Architecture](https://mlsanity.com/paper/2601.00806/) (2025)
- [Neuromorphic Object Detection: An In-Depth Study and Future Directions](https://mlsanity.com/paper/2607.23576/) (2026)
- [BLADE: ReliaBle Dynamic Hardware-Aware SNN-ANN Boundary SeLection for Event-BAseD Object DEtection](https://mlsanity.com/paper/2609.17562/) (2026)

## Abstract (arXiv)

Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames and yield sparse, energy-efficient encodings of scenes, in addition to low latency, high dynamic range, and lack of motion blur. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks, trained with backpropagation. However, using these approaches for event streams requires a transformation to a synchronous paradigm, which not only loses computational efficiency, but also misses opportunities to extract spatio-temporal features. In this article we propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection, combining a spiking neural network (SNN) backbone for efficient event-based feature extraction, and a subsequent analog neural network (ANN) head to solve synchronous classification and detection tasks. This is achieved by combining standard backpropagation with surrogate gradient training to propagate gradients through the SNN. Hybrid SNN-ANNs can be trained without conversion, and result in highly accurate networks that are substantially more computationally efficient than their ANN counterparts. We demonstrate results on event-based classification and object detection datasets, in which only the architecture of the ANN heads need to be adapted to the tasks, and no conversion of the event-based input is necessary. Since ANNs and SNNs require different hardware paradigms to maximize their efficiency, we envision that SNN backbone and ANN head can be executed on different processing units, and thus analyze the necessary bandwidth to communicate between the two parts. Hybrid networks are promising architectures to further advance machine learning approaches for event-based vision, without having to compromise on efficiency.