# P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications

arXiv: https://arxiv.org/abs/2203.04737 · HTML: https://mlsanity.com/paper/2203.04737/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Understanding Batch Normalization](https://mlsanity.com/paper/1806.02375/) (2018)
- [Benchmarking TinyML Systems: Challenges and Direction](https://mlsanity.com/paper/2003.04821/) (2020)
- [HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise](https://mlsanity.com/paper/2110.11417/) (2021)
- [The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence](https://mlsanity.com/paper/2002.04806/) (2020)
- [Visual Wake Words Dataset](https://mlsanity.com/paper/1906.05721/) (2019)
- [Reconfiguring the Imaging Pipeline for Computer Vision](https://mlsanity.com/paper/1705.04352/) (2017)
- [Towards Automatic Wild Animal Monitoring: Identification of Animal Species in Camera-trap Images using Very Deep Convolutional Neural Networks](https://mlsanity.com/paper/1603.06169/) (2016)
- [RNNPool: Efficient Non-linear Pooling for RAM Constrained Inference](https://mlsanity.com/paper/2002.11921/) (2020)
- [RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars](https://mlsanity.com/paper/1809.00072/) (2018)
- [Pre-defined Sparsity for Low-Complexity Convolutional Neural Networks](https://mlsanity.com/paper/2001.10710/) (2020)
- [Fully Embedding Fast Convolutional Networks on Pixel Processor Arrays](https://mlsanity.com/paper/2004.12525/) (2020)
- [Training Energy-Efficient Deep Spiking Neural Networks with Single-Spike Hybrid Input Encoding](https://mlsanity.com/paper/2107.12374/) (2021)
- [Can Deep Neural Networks be Converted to Ultra Low-Latency Spiking Neural Networks?](https://mlsanity.com/paper/2112.12133/) (2021)
- [AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator](https://mlsanity.com/paper/2111.06503/) (2021)
- [IMAC: In-memory multi-bit Multiplication andACcumulation in 6T SRAM Array](https://mlsanity.com/paper/2003.12558/) (2020)
- [Fundamental Limits on Energy-Delay-Accuracy of In-memory Architectures in Inference Applications](https://mlsanity.com/paper/2012.13645/) (2020)
- [PISA: A Binary-Weight Processing-In-Sensor Accelerator for Edge Image Processing](https://mlsanity.com/paper/2202.09035/) (2022)
- [HYPER-SNN: Towards Energy-efficient Quantized Deep Spiking Neural Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2107.11979/) (2021)
- [A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture](https://mlsanity.com/paper/2101.03308/) (2021)

## Built on by (most cited first, 16 of 16)

- [Self-Attentive Pooling for Efficient Deep Learning](https://mlsanity.com/paper/2209.07659/) (2022)
- [In-Sensor & Neuromorphic Computing are all you need for Energy Efficient Computer Vision](https://mlsanity.com/paper/2212.10881/) (2022)
- [MaskVD: Region Masking for Efficient Video Object Detection](https://mlsanity.com/paper/2407.12067/) (2024)
- [Toward Efficient Hyperspectral Image Processing inside Camera Pixels](https://mlsanity.com/paper/2203.05696/) (2022)
- [Hoyer regularizer is all you need for ultra low-latency spiking neural networks](https://mlsanity.com/paper/2212.10170/) (2022)
- [TinyReptile: TinyML with Federated Meta-Learning](https://mlsanity.com/paper/2304.05201/) (2023)
- [A Review on Digital Pixel Sensors](https://mlsanity.com/paper/2402.04507/) (2024)
- [End-to-End Efficiency in Keyword Spotting: A System-Level Approach for Embedded Microcontrollers](https://mlsanity.com/paper/2509.07051/) (2025)
- [In-Pixel Foreground and Contrast Enhancement Circuits with Customizable Mapping](https://mlsanity.com/paper/2410.18052/) (2024)
- [Enabling ISP-less Low-Power Computer Vision](https://mlsanity.com/paper/2210.05451/) (2022)
- [Let's Roll: Synthetic Dataset Analysis for Pedestrian Detection Across Different Shutter Types](https://mlsanity.com/paper/2309.08136/) (2023)
- [Sense, Predict, Adapt, Repeat: A Blueprint for Design of New Adaptive AI-Centric Sensing Systems](https://mlsanity.com/paper/2312.07602/) (2023)
- [Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors](https://mlsanity.com/paper/2409.17341/) (2024)
- [ReDistill: Residual Encoded Distillation for Peak Memory Reduction of CNNs](https://mlsanity.com/paper/2406.03744/) (2024)
- [STEMPix: A Phase-Transition-Material-Based Pixel Sensor for Resolving Edge-Movement Direction](https://mlsanity.com/paper/2609.04435/) (2026)
- [Hardware-Aware Learned Representation Compression for Distributed In-Sensor Vision](https://mlsanity.com/paper/2609.13947/) (2026)

## Abstract (arXiv)

The demand to process vast amounts of data generated from state-of-the-art high resolution cameras has motivated novel energy-efficient on-device AI solutions. Visual data in such cameras are usually captured in the form of analog voltages by a sensor pixel array, and then converted to the digital domain for subsequent AI processing using analog-to-digital converters (ADC). Recent research has tried to take advantage of massively parallel low-power analog/digital computing in the form of near- and in-sensor processing, in which the AI computation is performed partly in the periphery of the pixel array and partly in a separate on-board CPU/accelerator. Unfortunately, high-resolution input images still need to be streamed between the camera and the AI processing unit, frame by frame, causing energy, bandwidth, and security bottlenecks. To mitigate this problem, we propose a novel Processing-in-Pixel-in-memory (P2M) paradigm, that customizes the pixel array by adding support for analog multi-channel, multi-bit convolution, batch normalization, and ReLU (Rectified Linear Units). Our solution includes a holistic algorithm-circuit co-design approach and the resulting P2M paradigm can be used as a drop-in replacement for embedding memory-intensive first few layers of convolutional neural network (CNN) models within foundry-manufacturable CMOS image sensor platforms. Our experimental results indicate that P2M reduces data transfer bandwidth from sensors and analog to digital conversions by ~21x, and the energy-delay product (EDP) incurred in processing a MobileNetV2 model on a TinyML use case for visual wake words dataset (VWW) by up to ~11x compared to standard near-processing or in-sensor implementations, without any significant drop in test accuracy.