mlsanityData to 7 Oct 2026

Toward Efficient Hyperspectral Image Processing inside Camera Pixels

arXiv 2203.05696 ↗First posted 11 Mar 2022eess.IV cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Hyperspectral cameras generate a large amount of data due to the presence of hundreds of spectral bands as opposed to only three channels (red, green, and blue) in traditional cameras. This requires a significant amount of data transmission between the hyperspectral image sensor and a processor used to classify/detect/track the images, frame by frame, expending high energy and causing bandwidth and security bottlenecks. To mitigate this problem, we propose a form of processing-in-pixel (PIP) that leverages advanced CMOS technologies to enable the pixel array to perform a wide range of complex operations required by the modern convolutional neural networks (CNN) for hyperspectral image recognition (HSI). Consequently, our PIP-optimized custom CNN layers effectively compress the input data, significantly reducing the bandwidth required to transmit the data downstream to the HSI processing unit. This reduces the average energy consumption associated with pixel array of cameras and the CNN processing unit by 25.06x and 3.90x respectively, compared to existing hardware implementations. Our custom models yield average test accuracies within 0.56% of the baseline models for the standard HSI benchmarks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

13 of 28 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
  3. 2019HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classificationcited by 91
  4. 2018Three dimensional Deep Learning approach for remote sensing image classificationcited by 50
  5. 2019Visual Wake Words Datasetcited by 46
  6. 2020FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classificationcited by 27
  7. 2022P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applicationscited by 16
  8. 2018HSI-CNN: A Novel Convolution Neural Network for Hyperspectral Imagecited by 14
  9. 2020Fully Embedding Fast Convolutional Networks on Pixel Processor Arrayscited by 13
  10. 2021Training Energy-Efficient Deep Spiking Neural Networks with Single-Spike Hybrid Input Encodingcited by 10
  11. 2021Can Deep Neural Networks be Converted to Ultra Low-Latency Spiking Neural Networks?cited by 9
  12. 2020Fundamental Limits on Energy-Delay-Accuracy of In-memory Architectures in Inference Applicationscited by 3
1 more
  1. 2021HYPER-SNN: Towards Energy-efficient Quantized Deep Spiking Neural Networks for Hyperspectral Image Classificationcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 4.

4 papers.

  1. 2022Self-Attentive Pooling for Efficient Deep Learningcited by 8
  2. 2022In-Sensor & Neuromorphic Computing are all you need for Energy Efficient Computer Visioncited by 6
  3. 2022Hoyer regularizer is all you need for ultra low-latency spiking neural networkscited by 4
  4. 2022Enabling ISP-less Low-Power Computer Visioncited by 1