# Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline

arXiv: https://arxiv.org/abs/2304.06793 · HTML: https://mlsanity.com/paper/2304.06793/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 124 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 30 of 32)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [CondConv: Conditionally Parameterized Convolutions for Efficient Inference](https://mlsanity.com/paper/1904.04971/) (2019)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Attention Spiking Neural Networks](https://mlsanity.com/paper/2209.13929/) (2022)
- [Temporal-wise Attention Spiking Neural Networks for Event Streams Classification](https://mlsanity.com/paper/2107.11711/) (2021)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)](https://mlsanity.com/paper/1708.04198/) (2017)
- [Brain-inspired computing: We need a master plan](https://mlsanity.com/paper/2104.14517/) (2021)
- [HFirst: A Temporal Approach to Object Recognition](https://mlsanity.com/paper/1508.01176/) (2015)
- [Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks](https://mlsanity.com/paper/2007.01204/) (2020)
- [HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors](https://mlsanity.com/paper/2211.09648/) (2022)
- [Glance and Focus Networks for Dynamic Visual Recognition](https://mlsanity.com/paper/2201.03014/) (2022)
- [Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences](https://mlsanity.com/paper/2005.02183/) (2020)
- [Optimizing the energy consumption of spiking neural networks for neuromorphic applications](https://mlsanity.com/paper/1912.01268/) (2019)
- [TAda! Temporally-Adaptive Convolutions for Video Understanding](https://mlsanity.com/paper/2110.06178/) (2021)
- [Survey of Machine Learning Accelerators](https://mlsanity.com/paper/2009.00993/) (2020)

## Built on by (most cited first, 30 of 52)

- [Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training](https://mlsanity.com/paper/2411.16061/) (2024)
- [SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms](https://mlsanity.com/paper/2406.03287/) (2024)
- [MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map](https://mlsanity.com/paper/2411.10741/) (2024)
- [SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models](https://mlsanity.com/paper/2408.14909/) (2024)
- [Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation](https://mlsanity.com/paper/2412.14587/) (2024)
- [Efficient 3D Recognition with Event-driven Spike Sparse Convolution](https://mlsanity.com/paper/2412.07360/) (2024)
- [Differential Coding for Training-Free ANN-to-SNN Conversion](https://mlsanity.com/paper/2503.00301/) (2025)
- [Neural Brain: A Neuroscience-inspired Framework for Embodied Agents](https://mlsanity.com/paper/2505.07634/) (2025)
- [Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients](https://mlsanity.com/paper/2503.03272/) (2025)
- [Drone Detection with Event Cameras](https://mlsanity.com/paper/2508.04564/) (2025)
- [RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Coding](https://mlsanity.com/paper/2407.20099/) (2024)
- [Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks](https://mlsanity.com/paper/2503.04223/) (2025)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Representation](https://mlsanity.com/paper/2504.06748/) (2025)
- [SpikingBrain: Spiking Brain-inspired Large Models](https://mlsanity.com/paper/2509.05276/) (2025)
- [SpikeVideoFormer: An Efficient Spike-Driven Video Transformer with Hamming Attention and O(T) Complexity](https://mlsanity.com/paper/2505.10352/) (2025)
- [SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network](https://mlsanity.com/paper/2408.00788/) (2024)
- [Exploring the Limitations of Layer Synchronization in Spiking Neural Networks](https://mlsanity.com/paper/2408.05098/) (2024)
- [Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Review](https://mlsanity.com/paper/2504.08588/) (2025)
- [Event2Vec: Processing Neuromorphic Events Directly by Representations in Vector Space](https://mlsanity.com/paper/2504.15371/) (2025)
- [Temporal Reversal Regularization for Spiking Neural Networks: Hybrid Spatio-Temporal Invariance for Generalization](https://mlsanity.com/paper/2408.09108/) (2024)
- [FAS: Fast ANN-SNN Conversion for Spiking Large Language Models](https://mlsanity.com/paper/2502.04405/) (2025)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [Drone Detection Using a Low-Power Neuromorphic Virtual Tripwire](https://mlsanity.com/paper/2509.12997/) (2025)
- [Efficient Spike-driven Transformer for High-performance Drone-View Geo-Localization](https://mlsanity.com/paper/2512.19365/) (2025)
- [Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks](https://mlsanity.com/paper/2603.13283/) (2026)
- [SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference](https://mlsanity.com/paper/2604.22575/) (2026)
- [PASS: Path-selective State Space Model for Event-based Recognition](https://mlsanity.com/paper/2409.16953/) (2024)
- [STOP: Spatiotemporal Orthogonal Propagation for Weight-Threshold-Leakage Synergistic Training of Deep Spiking Neural Networks](https://mlsanity.com/paper/2411.11082/) (2024)
- [On the Privacy Risks of Spiking Neural Networks: A Membership Inference Analysis](https://mlsanity.com/paper/2502.13191/) (2025)

## Abstract (arXiv)

Edge computing solutions that enable the extraction of high-level information from a variety of sensors is in increasingly high demand. This is due to the increasing number of smart devices that require sensory processing for their application on the edge. To tackle this problem, we present a smart vision sensor System on Chip (SoC), featuring an event-based camera and a low-power asynchronous spiking Convolutional Neural Network (sCNN) computing architecture embedded on a single chip. By combining both sensor and processing on a single die, we can lower unit production costs significantly. Moreover, the simple end-to-end nature of the SoC facilitates small stand-alone applications as well as functioning as an edge node in larger systems. The event-driven nature of the vision sensor delivers high-speed signals in a sparse data stream. This is reflected in the processing pipeline, which focuses on optimising highly sparse computation and minimising latency for 9 sCNN layers to 3.36μs for an incoming event. Overall, this results in an extremely low-latency visual processing pipeline deployed on a small form factor with a low energy budget and sensor cost. We present the asynchronous architecture, the individual blocks, and the sCNN processing principle and benchmark against other sCNN capable processors.