# GANESH: Generalizable NeRF for Lensless Imaging

arXiv: https://arxiv.org/abs/2411.04810 · HTML: https://mlsanity.com/paper/2411.04810/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields](https://mlsanity.com/paper/2106.13228/) (2021)
- [Is Attention All That NeRF Needs?](https://mlsanity.com/paper/2207.13298/) (2022)
- [Ev-NeRF: Event Based Neural Radiance Field](https://mlsanity.com/paper/2206.12455/) (2022)
- [DiffuserCam: Lensless Single-exposure 3D Imaging](https://mlsanity.com/paper/1710.02134/) (2017)
- [FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements](https://mlsanity.com/paper/2010.15440/) (2020)
- [Thermal-NeRF: Neural Radiance Fields from an Infrared Camera](https://mlsanity.com/paper/2403.10340/) (2024)
- [Hyperspectral Neural Radiance Fields](https://mlsanity.com/paper/2403.14839/) (2024)
- [X-NeRF: Explicit Neural Radiance Field for Multi-Scene 360circ  Insufficient RGB-D Views](https://mlsanity.com/paper/2210.05135/) (2022)
- [CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Rendering](https://mlsanity.com/paper/2311.15510/) (2023)
- [Lensless Photography with only an image sensor](https://mlsanity.com/paper/1702.06619/) (2017)
- [LPSNet: End-to-End Human Pose and Shape Estimation with Lensless Imaging](https://mlsanity.com/paper/2404.01941/) (2024)
- [Radiance Fields from Photons](https://mlsanity.com/paper/2407.09386/) (2024)
- [Compressive lensless endoscopy with partial speckle scanning](https://mlsanity.com/paper/2104.10959/) (2021)

## Built on by (most cited first, 2 of 2)

- [PhotonSplat: 3D Scene Reconstruction and Colorization from SPAD Sensors](https://mlsanity.com/paper/2506.21680/) (2025)
- [PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty](https://mlsanity.com/paper/2604.17831/) (2026)

## Abstract (arXiv)

Lensless imaging offers a significant opportunity to develop ultra-compact cameras by removing the conventional bulky lens system. However, without a focusing element, the sensor's output is no longer a direct image but a complex multiplexed scene representation. Traditional methods have attempted to address this challenge by employing learnable inversions and refinement models, but these methods are primarily designed for 2D reconstruction and do not generalize well to 3D reconstruction. We introduce GANESH, a novel framework designed to enable simultaneous refinement and novel view synthesis from multi-view lensless images. Unlike existing methods that require scene-specific training, our approach supports on-the-fly inference without retraining on each scene. Moreover, our framework allows us to tune our model to specific scenes, enhancing the rendering and refinement quality. To facilitate research in this area, we also present the first multi-view lensless dataset, LenslessScenes. Extensive experiments demonstrate that our method outperforms current approaches in reconstruction accuracy and refinement quality. Code and video results are available at https://rakesh-123-cryp.github.io/Rakesh.github.io/