# Fully Embedding Fast Convolutional Networks on Pixel Processor Arrays

arXiv: https://arxiv.org/abs/2004.12525 · HTML: https://mlsanity.com/paper/2004.12525/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2020.

## Built on by (most cited first, 13 of 13)

- [Time-Ordered Recent Event (TORE) Volumes for Event Cameras](https://mlsanity.com/paper/2103.06108/) (2021)
- [P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications](https://mlsanity.com/paper/2203.04737/) (2022)
- [In-Sensor & Neuromorphic Computing are all you need for Energy Efficient Computer Vision](https://mlsanity.com/paper/2212.10881/) (2022)
- [PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors](https://mlsanity.com/paper/2304.05440/) (2023)
- [Toward Efficient Hyperspectral Image Processing inside Camera Pixels](https://mlsanity.com/paper/2203.05696/) (2022)
- [A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture](https://mlsanity.com/paper/2101.03308/) (2021)
- [Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retina](https://mlsanity.com/paper/2504.13457/) (2025)
- [Direct Servo Control from In-Sensor CNN Inference with A Pixel Processor Array](https://mlsanity.com/paper/2106.07561/) (2021)
- [Enabling ISP-less Low-Power Computer Vision](https://mlsanity.com/paper/2210.05451/) (2022)
- [Sense, Predict, Adapt, Repeat: A Blueprint for Design of New Adaptive AI-Centric Sensing Systems](https://mlsanity.com/paper/2312.07602/) (2023)
- [On-Sensor Binarized Fully Convolutional Neural Network with A Pixel Processor Array](https://mlsanity.com/paper/2202.00836/) (2022)
- [FrequencyFormer: A Co-Designed Sensor-to-Processor Pipeline for Frequency-Domain Vision Transformer Inference](https://mlsanity.com/paper/2606.19574/) (2026)
- [Hardware-Aware Learned Representation Compression for Distributed In-Sensor Vision](https://mlsanity.com/paper/2609.13947/) (2026)

## Abstract (arXiv)

We present a novel method of CNN inference for pixel processor array (PPA) vision sensors, designed to take advantage of their massive parallelism and analog compute capabilities. PPA sensors consist of an array of processing elements (PEs), with each PE capable of light capture, data storage and computation, allowing various computer vision processing to be executed directly upon the sensor device. The key idea behind our approach is storing network weights "in-pixel" within the PEs of the PPA sensor itself to allow various computations, such as multiple different image convolutions, to be carried out in parallel. Our approach can perform convolutional layers, max pooling, ReLu, and a final fully connected layer entirely upon the PPA sensor, while leaving no untapped computational resources. This is in contrast to previous works that only use a sensor-level processing to sequentially compute image convolutions, and must transfer data to an external digital processor to complete the computation. We demonstrate our approach on the SCAMP-5 vision system, performing inference of a MNIST digit classification network at over 3000 frames per second and over 93% classification accuracy. This is the first work demonstrating CNN inference conducted entirely upon the processor array of a PPA vision sensor device, requiring no external processing.