# Depth from Coupled Optical Differentiation

arXiv: https://arxiv.org/abs/2409.10725 · HTML: https://mlsanity.com/paper/2409.10725/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 45 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 7 of 7)

- [Synthetic Depth-of-Field with a Single-Camera Mobile Phone](https://mlsanity.com/paper/1806.04171/) (2018)
- [Deep Optics for Monocular Depth Estimation and 3D Object Detection](https://mlsanity.com/paper/1904.08601/) (2019)
- [Single Image Depth Estimation Trained via Depth from Defocus Cues](https://mlsanity.com/paper/2001.05036/) (2020)
- [Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset](https://mlsanity.com/paper/1807.10376/) (2018)
- [An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies](https://mlsanity.com/paper/2012.06772/) (2020)
- [Computer Stereo Vision for Autonomous Driving](https://mlsanity.com/paper/2012.03194/) (2020)
- [Physical Cue based Depth-Sensing by Color Coding with Deaberration Network](https://mlsanity.com/paper/1908.00329/) (2019)

## Built on by (most cited first, 2 of 2)

- [Blurry-Edges: Photon-Limited Depth Estimation from Defocused Boundaries](https://mlsanity.com/paper/2503.23606/) (2025)
- [Focal Split: Untethered Snapshot Depth from Differential Defocus](https://mlsanity.com/paper/2504.11202/) (2025)

## Abstract (arXiv)

We propose depth from coupled optical differentiation, a low-computation passive-lighting 3D sensing mechanism. It is based on our discovery that per-pixel object distance can be rigorously determined by a coupled pair of optical derivatives of a defocused image using a simple, closed-form relationship. Unlike previous depth-from-defocus (DfD) methods that leverage spatial derivatives of the image to estimate scene depths, the proposed mechanism's use of only optical derivatives makes it significantly more robust to noise. Furthermore, unlike many previous DfD algorithms with requirements on aperture code, this relationship is proved to be universal to a broad range of aperture codes. We build the first 3D sensor based on depth from coupled optical differentiation. Its optical assembly includes a deformable lens and a motorized iris, which enables dynamic adjustments to the optical power and aperture radius. The sensor captures two pairs of images: one pair with a differential change of optical power and the other with a differential change of aperture scale. From the four images, a depth and confidence map can be generated with only 36 floating point operations per output pixel (FLOPOP), more than ten times lower than the previous lowest passive-lighting depth sensing solution to our knowledge. Additionally, the depth map generated by the proposed sensor demonstrates more than twice the working range of previous DfD methods while using significantly lower computation.