# Distinguishing Refracted Features using Light Field Cameras with Application to Structure from Motion

arXiv: https://arxiv.org/abs/1806.07375 · HTML: https://mlsanity.com/paper/1806.07375/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2018.

## Built on by (most cited first, 4 of 4)

- [Deep Learning in Concealed Dense Prediction](https://mlsanity.com/paper/2504.10979/) (2025)
- [Segment Anything in Light Fields for Real-Time Applications via Constrained Prompting](https://mlsanity.com/paper/2411.13840/) (2024)
- [PlenoCI: Plenoptic CharacterIstics for View Dependence Aware Change Classification](https://mlsanity.com/paper/2609.28930/) (2026)
- [Refractive Light-Field Features for Curved Transparent Objects in Structure from Motion](https://mlsanity.com/paper/2103.15349/) (2021)

## Abstract (arXiv)

Robots must reliably interact with refractive objects in many applications; however, refractive objects can cause many robotic vision algorithms to become unreliable or even fail, particularly feature-based matching applications, such as structure-from-motion. We propose a method to distinguish between refracted and Lambertian image features using a light field camera. Specifically, we propose to use textural cross-correlation to characterise apparent feature motion in a single light field, and compare this motion to its Lambertian equivalent based on 4D light field geometry. Our refracted feature distinguisher has a 34.3% higher rate of detection compared to state-of-the-art for light fields captured with large baselines relative to the refractive object. Our method also applies to light field cameras with much smaller baselines than previously considered, yielding up to 2 times better detection for 2D-refractive objects, such as a sphere, and up to 8 times better for 1D-refractive objects, such as a cylinder. For structure from motion, we demonstrate that rejecting refracted features using our distinguisher yields up to 42.4% lower reprojection error, and lower failure rate when the robot is approaching refractive objects. Our method lead to more robust robot vision in the presence of refractive objects.