# Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion

arXiv: https://arxiv.org/abs/2412.02798 · HTML: https://mlsanity.com/paper/2412.02798/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2304.12526/) (2023)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction](https://mlsanity.com/paper/2204.07908/) (2022)
- [HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging](https://mlsanity.com/paper/2203.02149/) (2022)
- [Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging](https://mlsanity.com/paper/2205.10102/) (2022)
- [Deep Gaussian Scale Mixture Prior for Spectral Compressive Imaging](https://mlsanity.com/paper/2103.07152/) (2021)
- [HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models](https://mlsanity.com/paper/2402.15865/) (2024)
- [Spatial-Spectral Transformer for Hyperspectral Image Denoising](https://mlsanity.com/paper/2211.14090/) (2022)
- [SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models](https://mlsanity.com/paper/2304.05961/) (2023)
- [Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array](https://mlsanity.com/paper/2006.08565/) (2020)
- [Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problems](https://mlsanity.com/paper/2406.02462/) (2024)
- [IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models](https://mlsanity.com/paper/2410.21759/) (2024)
- [In2SET: Intra-Inter Similarity Exploiting Transformer for Dual-Camera Compressive Hyperspectral Imaging](https://mlsanity.com/paper/2312.13319/) (2023)
- [Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems](https://mlsanity.com/paper/2410.11730/) (2024)
- [Aperture Diffraction for Compact Snapshot Spectral Imaging](https://mlsanity.com/paper/2309.16372/) (2023)
- [DiffSCI: Zero-Shot Snapshot Compressive Imaging via Iterative Spectral Diffusion Model](https://mlsanity.com/paper/2311.11417/) (2023)
- [Multistable Shape from Shading Emerges from Patch Diffusion](https://mlsanity.com/paper/2405.14530/) (2024)
- [Limitations of Data-Driven Spectral Reconstruction -- An Optics-Aware Analysis](https://mlsanity.com/paper/2401.03835/) (2024)
- [Polarization Multi-Image Synthesis with Birefringent Metasurfaces](https://mlsanity.com/paper/2307.08106/) (2023)

## Built on by (most cited first, 1 of 1)

- [Uncertainty Quantification in HSI Reconstruction using Physics-Aware Diffusion Priors and Optics-Encoded Measurements](https://mlsanity.com/paper/2511.18473/) (2025)

## Abstract (arXiv)

We consider the problem of reconstructing a HxWx31 hyperspectral image from a H× W grayscale snapshot measurement that is captured using only a single diffractive lens and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present a model that produces high-quality results in simulation and experiment. We make efficient use of limited training data by creating a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance by physical consistency with the system's optical point spread function. Our experiments reveal that the patch size can be as small as the point spread function, with local optical cues being the main source of information about complete spectra. Also, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error.