# PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging

arXiv: https://arxiv.org/abs/2409.17996 · HTML: https://mlsanity.com/paper/2409.17996/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [DiffuserCam: Lensless Single-exposure 3D Imaging](https://mlsanity.com/paper/1710.02134/) (2017)
- [Learned reconstructions for practical mask-based lensless imaging](https://mlsanity.com/paper/1908.11502/) (2019)
- [FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements](https://mlsanity.com/paper/2010.15440/) (2020)
- [Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy](https://mlsanity.com/paper/2010.05382/) (2020)
- [Aberration-Aware Depth-from-Focus](https://mlsanity.com/paper/2303.04654/) (2023)
- [Unrolled Primal-Dual Networks for Lensless Cameras](https://mlsanity.com/paper/2203.04353/) (2022)
- [Privacy-Enhancing Optical Embeddings for Lensless Classification](https://mlsanity.com/paper/2211.12864/) (2022)
- [Angle Sensitive Pixels for Lensless Imaging on Spherical Sensors](https://mlsanity.com/paper/2306.15953/) (2023)

## Built on by (most cited first, 13 of 13)

- [Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution](https://mlsanity.com/paper/2501.11561/) (2025)
- [DetailGen3D: Generative 3D Geometry Enhancement via Data-Dependent Flow](https://mlsanity.com/paper/2411.16820/) (2024)
- [LensNet: An End-to-End Learning Framework for Empirical Point Spread Function Modeling and Lensless Imaging Reconstruction](https://mlsanity.com/paper/2505.01755/) (2025)
- [Towards Robust and Generalizable Lensless Imaging with Modular Learned Reconstruction](https://mlsanity.com/paper/2502.01102/) (2025)
- [Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models](https://mlsanity.com/paper/2508.07818/) (2025)
- [Twins: Learn to Predict Unified Representations with Focal Loss](https://mlsanity.com/paper/2607.22531/) (2026)
- [PreResQ-R1: Response-Preference Disentangled Ranking-and-Scoring Reinforcement Optimization for Robust Visual Quality Assessment](https://mlsanity.com/paper/2511.05393/) (2025)
- [Null-Space Diffusion Distillation Unlocks Speed, Fidelity and Realism in Lensless Imaging](https://mlsanity.com/paper/2511.12024/) (2025)
- [A Lensless Polarization Camera](https://mlsanity.com/paper/2603.17156/) (2026)
- [Guided Lensless Polarization Imaging](https://mlsanity.com/paper/2603.27357/) (2026)
- [Integrated Forward-Inverse Network for Lensless Image Reconstruction](https://mlsanity.com/paper/2607.04608/) (2026)
- [LFD: Enabling Real-World Lensless Face Recognition with a Large-Scale Dataset](https://mlsanity.com/paper/2607.10094/) (2026)
- [Parallax Portrait Matting](https://mlsanity.com/paper/2607.11205/) (2026)

## Abstract (arXiv)

Lensless cameras offer significant advantages in size, weight, and cost compared to traditional lens-based systems. Without a focusing lens, lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, current algorithms struggle with inaccurate forward imaging models and insufficient priors to reconstruct high-quality images. To overcome these limitations, we introduce a novel two-stage approach for consistent and photorealistic lensless image reconstruction. The first stage of our approach ensures data consistency by focusing on accurately reconstructing the low-frequency content with a spatially varying deconvolution method that adjusts to changes in the Point Spread Function (PSF) across the camera's field of view. The second stage enhances photorealism by incorporating a generative prior from pre-trained diffusion models. By conditioning on the low-frequency content retrieved in the first stage, the diffusion model effectively reconstructs the high-frequency details that are typically lost in the lensless imaging process, while also maintaining image fidelity. Our method achieves a superior balance between data fidelity and visual quality compared to existing methods, as demonstrated with two popular lensless systems, PhlatCam and DiffuserCam. Project website: https://phocolens.github.io/.