# A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception

arXiv: https://arxiv.org/abs/2303.14176 · HTML: https://mlsanity.com/paper/2303.14176/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 55 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2003.01811/) (2020)
- [Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation](https://mlsanity.com/paper/2005.01807/) (2020)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration](https://mlsanity.com/paper/2106.06984/) (2021)
- [Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach](https://mlsanity.com/paper/1704.02447/) (2017)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [Self-Supervised Learning of 3D Human Pose using Multi-view Geometry](https://mlsanity.com/paper/1903.02330/) (2019)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image](https://mlsanity.com/paper/1701.00295/) (2017)
- [Time-Ordered Recent Event (TORE) Volumes for Event Cameras](https://mlsanity.com/paper/2103.06108/) (2021)
- [EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera](https://mlsanity.com/paper/1908.11505/) (2019)
- [Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2006.01423/) (2020)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [EventHPE: Event-based 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2108.06819/) (2021)
- [Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning](https://mlsanity.com/paper/2109.01905/) (2021)
- [Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision](https://mlsanity.com/paper/2112.03423/) (2021)

## Built on by (most cited first, 13 of 13)

- [SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition](https://mlsanity.com/paper/2308.04369/) (2023)
- [Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention](https://mlsanity.com/paper/2403.10173/) (2024)
- [ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition](https://mlsanity.com/paper/2409.01564/) (2024)
- [ClearSight: Human Vision-Inspired Solutions for Event-Based Motion Deblurring](https://mlsanity.com/paper/2501.15808/) (2025)
- [Hybrid Layer-Wise ANN-SNN With Surrogate Spike Encoding-Decoding Structure](https://mlsanity.com/paper/2509.24411/) (2025)
- [FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Vision](https://mlsanity.com/paper/2603.19770/) (2026)
- [Asynchronous Bioplausible Neuron for SNN for Event Vision](https://mlsanity.com/paper/2311.11853/) (2023)
- [ParaRevSNN: A Parallel Reversible Spiking Neural Network for Efficient Training and Inference](https://mlsanity.com/paper/2508.01223/) (2025)
- [Energy-Aware Imitation Learning for Steering Prediction Using Events and Frames](https://mlsanity.com/paper/2603.28008/) (2026)
- [DBHN-Net: Dual-Branch Hybrid Neural Network For Low-Complexity Monaural Speech Enhancement](https://mlsanity.com/paper/2606.05911/) (2026)
- [FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection](https://mlsanity.com/paper/2606.17334/) (2026)
- [Neuromorphic Object Detection: An In-Depth Study and Future Directions](https://mlsanity.com/paper/2607.23576/) (2026)
- [NeuroFlex: Lossless Element-Level ANN-SNN Co-Execution for Efficient Sparse Inference](https://mlsanity.com/paper/2609.14092/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNN) are a class of bio-inspired neural networks that promise to bring low-power and low-latency inference to edge devices through asynchronous and sparse processing. However, being temporal models, SNNs depend heavily on expressive states to generate predictions on par with classical artificial neural networks (ANNs). These states converge only after long transient periods, and quickly decay without input data, leading to higher latency, power consumption, and lower accuracy. This work addresses this issue by initializing the state with an auxiliary ANN running at a low rate. The SNN then uses the state to generate predictions with high temporal resolution until the next initialization phase. Our hybrid ANN-SNN model thus combines the best of both worlds: It does not suffer from long state transients and state decay thanks to the ANN, and can generate predictions with high temporal resolution, low latency, and low power thanks to the SNN. We show for the task of event-based 2D and 3D human pose estimation that our method consumes 88% less power with only a 4% decrease in performance compared to its fully ANN counterparts when run at the same inference rate. Moreover, when compared to SNNs, our method achieves a 74% lower error. This research thus provides a new understanding of how ANNs and SNNs can be used to maximize their respective benefits.