mlsanityData to 7 Oct 2026

PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors

arXiv 2304.05440 ↗First posted 11 Apr 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Conventional image sensors digitize high-resolution images at fast frame rates, producing a large amount of data that needs to be transmitted off the sensor for further processing. This is challenging for perception systems operating on edge devices, because communication is power inefficient and induces latency. Fueled by innovations in stacked image sensor fabrication, emerging sensor-processors offer programmability and minimal processing capabilities directly on the sensor. We exploit these capabilities by developing an efficient recurrent neural network architecture, PixelRNN, that encodes spatio-temporal features on the sensor using purely binary operations. PixelRNN reduces the amount of data to be transmitted off the sensor by a factor of 64x compared to conventional systems while offering competitive accuracy for hand gesture recognition and lip reading tasks. We experimentally validate PixelRNN using a prototype implementation on the SCAMP-5 sensor-processor platform.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 39 of its 85 references that are in the map. First posted on arXiv 11 Apr 2023.

Builds on

39 of 85 references.

References · 39

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  7. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  8. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  9. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  10. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  11. 2019Event-based Vision: A Surveycited by 1,010
  12. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
27 more
  1. 2020What is the State of Neural Network Pruning?cited by 578
  2. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  3. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  4. 2013Do Deep Nets Really Need to be Deep?cited by 466
  5. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  6. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  7. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
  8. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 223
  9. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  10. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
  11. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  12. 2015Delving Deeper into Convolutional Networks for Learning Video Representationscited by 146
  13. 2020Lite Transformer with Long-Short Range Attentioncited by 127
  14. 2015Data-free parameter pruning for Deep Neural Networkscited by 112
  15. 2015Convolutional neural networks with low-rank regularizationcited by 102
  16. 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
  17. 2016SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Drivingcited by 55
  18. 2016Minimal Gated Unit for Recurrent Neural Networkscited by 40
  19. 2019Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networkscited by 30
  20. 2020Fully Embedding Fast Convolutional Networks on Pixel Processor Arrayscited by 13
  21. 2022SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systemscited by 13
  22. 2022Learning Spatially Varying Pixel Exposures for Motion Deblurringcited by 12
  23. 2021MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encodingcited by 10
  24. 2019A Camera That CNNs: Towards Embedded Neural Networks on Pixel Processor Arrayscited by 9
  25. 2022Distributed On-Sensor Compute System for AR/VR Devices: A Semi-Analytical Simulation Framework for Power Estimationcited by 8
  26. 2020AnalogNet: Convolutional Neural Network Inference on Analog Focal Plane Sensor Processorscited by 4
  27. 2021Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processorscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 2; 2026: 3.

6 papers.

  1. 2025Generative Models in Decision Making: A Surveycited by 14
  2. 2023Neural Network Methods for Radiation Detectors and Imagingcited by 2
  3. 2025Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retinacited by 2
  4. 2026PixVOD: Pixel-Distributed Direct Visual Odometry and Depth Estimationcited by 0
  5. 2026Programmable Silicon Retina on Pixel Processor Arraycited by 0
  6. 2026Rendering 3D Gaussians on a Graph Processorcited by 0