# A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

arXiv: https://arxiv.org/abs/2101.03308 · HTML: https://mlsanity.com/paper/2101.03308/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 45 references that are in the map.
- First posted on arXiv 9 Jan 2021.

## Builds on (most cited first, 7 of 7)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Trunk-Branch Ensemble Convolutional Neural Networks for Video-based Face Recognition](https://mlsanity.com/paper/1607.05427/) (2016)
- [GAC-GAN: A General Method for Appearance-Controllable Human Video Motion Transfer](https://mlsanity.com/paper/1911.10672/) (2019)
- [Fully Embedding Fast Convolutional Networks on Pixel Processor Arrays](https://mlsanity.com/paper/2004.12525/) (2020)
- [Energy-Efficient Hybrid Stochastic-Binary Neural Networks for Near-Sensor Computing](https://mlsanity.com/paper/1706.02344/) (2017)
- [A Foreground-background Parallel Compression with Residual Encoding for Surveillance Video](https://mlsanity.com/paper/2001.06590/) (2020)

## Built on by (most cited first, 2 of 2)

- [P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications](https://mlsanity.com/paper/2203.04737/) (2022)
- [HiRISE: High-Resolution Image Scaling for Edge ML via In-Sensor Compression and Selective ROI](https://mlsanity.com/paper/2408.03956/) (2024)

## Abstract (arXiv)

The separation of the data capture and analysis in modern vision systems has led to a massive amount of data transfer between the end devices and cloud computers, resulting in long latency, slow response, and high power consumption. Efficient hardware architectures are under focused development to enable Artificial Intelligence (AI) at the resource-limited end sensing devices. One of the most promising solutions is to enable Processing-in-Pixel (PIP) scheme. However, the conventional schemes suffer from the low fill-factor issue. This paper proposes a PIP based CMOS sensor architecture, which allows convolution operation before the column readout circuit to significantly improve the image reading speed with much lower power consumption. The simulation results show that the proposed architecture could support the computing efficiency up to 11.65 TOPS/W at the 8-bit weight configuration, which is three times as high as the conventional schemes. The transistors required for each pixel are only 2.5T, significantly improving the fill-factor.