# Single-Shot Metric Depth from Focused Plenoptic Cameras

arXiv: https://arxiv.org/abs/2412.02386 · HTML: https://mlsanity.com/paper/2412.02386/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 61 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth](https://mlsanity.com/paper/1904.02028/) (2019)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency](https://mlsanity.com/paper/2011.14272/) (2020)
- [LightDepth: Single-View Depth Self-Supervision from Illumination Decline](https://mlsanity.com/paper/2308.10525/) (2023)
- [LiFF: Light Field Features in Scale and Depth](https://mlsanity.com/paper/1901.03916/) (2019)
- [Towards a quality metric for dense light fields](https://mlsanity.com/paper/1704.07576/) (2017)
- [DepthNet: Real-Time LiDAR Point Cloud Depth Completion for Autonomous Vehicles](https://mlsanity.com/paper/2007.02438/) (2020)
- [SfM-TTR: Using Structure from Motion for Test-Time Refinement of Single-View Depth Networks](https://mlsanity.com/paper/2211.13551/) (2022)
- [The Third Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2404.16831/) (2024)
- [PlenoptiCam v1.0: A light-field imaging framework](https://mlsanity.com/paper/2010.11687/) (2020)
- [Baseline and Triangulation Geometry in a Standard Plenoptic Camera](https://mlsanity.com/paper/2010.04638/) (2020)
- [3D Face Reconstruction from Light Field Images: A Model-free Approach](https://mlsanity.com/paper/1711.05953/) (2017)
- [Partial light field tomographic reconstruction from a fixed-camera focal stack](https://mlsanity.com/paper/1503.01903/) (2015)
- [A Model for Multi-View Residual Covariances based on Perspective Deformation](https://mlsanity.com/paper/2202.00765/) (2022)

## Built on by (most cited first, 1 of 1)

- [PRISM-VO: Scale-Aware Visual Odometry Using Photometric Plenoptic Bundle Adjustment](https://mlsanity.com/paper/2607.00176/) (2026)

## Abstract (arXiv)

Metric depth estimation from visual sensors is crucial for robots to perceive, navigate, and interact with their environment. Traditional range imaging setups, such as stereo or structured light cameras, face hassles including calibration, occlusions, and hardware demands, with accuracy limited by the baseline between cameras. Single- and multi-view monocular depth offers a more compact alternative, but is constrained by the unobservability of the metric scale. Light field imaging provides a promising solution for estimating metric depth by using a unique lens configuration through a single device. However, its application to single-view dense metric depth is under-addressed mainly due to the technology's high cost, the lack of public benchmarks, and proprietary geometrical models and software. Our work explores the potential of focused plenoptic cameras for dense metric depth. We propose a novel pipeline that predicts metric depth from a single plenoptic camera shot by first generating a sparse metric point cloud using machine learning, which is then used to scale and align a dense relative depth map regressed by a foundation depth model, resulting in dense metric depth. To validate it, we curated the Light Field & Stereo Image Dataset (LFS) of real-world light field images with stereo depth labels, filling a current gap in existing resources. Experimental results show that our pipeline produces accurate metric depth predictions, laying a solid groundwork for future research in this field.