# Learning to See Through Obstructions

arXiv: https://arxiv.org/abs/2004.01180 · HTML: https://mlsanity.com/paper/2004.01180/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 30 of 38)

- [Removing Objects From Neural Radiance Fields](https://mlsanity.com/paper/2212.11966/) (2022)
- [Single Image Reflection Separation via Component Synergy](https://mlsanity.com/paper/2308.10027/) (2023)
- [Generative Omnimatte: Learning to Decompose Video into Layers](https://mlsanity.com/paper/2411.16683/) (2024)
- [Robust Reflection Removal with Reflection-free Flash-only Cues](https://mlsanity.com/paper/2103.04273/) (2021)
- [GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models](https://mlsanity.com/paper/2404.07206/) (2024)
- [Temporally Consistent Semantic Video Editing](https://mlsanity.com/paper/2206.10590/) (2022)
- [Neural Image Representations for Multi-Image Fusion and Layer Separation](https://mlsanity.com/paper/2108.01199/) (2021)
- [Neural Spline Fields for Burst Image Fusion and Layer Separation](https://mlsanity.com/paper/2312.14235/) (2023)
- [Language-guided Image Reflection Separation](https://mlsanity.com/paper/2402.11874/) (2024)
- [DC-Gaussian: Improving 3D Gaussian Splatting for Reflective Dash Cam Videos](https://mlsanity.com/paper/2405.17705/) (2024)
- [Robust Reflection Removal with Flash-only Cues in the Wild](https://mlsanity.com/paper/2211.02914/) (2022)
- [DeNVeR: Deformable Neural Vessel Representations for Unsupervised Video Vessel Segmentation](https://mlsanity.com/paper/2406.01591/) (2024)
- [A Categorized Reflection Removal Dataset with Diverse Real-world Scenes](https://mlsanity.com/paper/2108.03380/) (2021)
- [Deep Image Matting: A Comprehensive Survey](https://mlsanity.com/paper/2304.04672/) (2023)
- [Removing Reflections from RAW Photos](https://mlsanity.com/paper/2404.14414/) (2024)
- [Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation](https://mlsanity.com/paper/2501.00637/) (2024)
- [Let's See Clearly: Contaminant Artifact Removal for Moving Cameras](https://mlsanity.com/paper/2104.08852/) (2021)
- [SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-Trained Siamese Transformers](https://mlsanity.com/paper/2112.09426/) (2021)
- [Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes](https://mlsanity.com/paper/2302.05916/) (2023)
- [Multi-view Self-supervised Disentanglement for General Image Denoising](https://mlsanity.com/paper/2309.05049/) (2023)
- [Stereo Waterdrop Removal with Row-wise Dilated Attention](https://mlsanity.com/paper/2108.03457/) (2021)
- [Seeing the World through Your Eyes](https://mlsanity.com/paper/2306.09348/) (2023)
- [FIRM: Flexible Interactive Reflection reMoval](https://mlsanity.com/paper/2406.01555/) (2024)
- [Reflection Removal through Efficient Adaptation of Diffusion Transformers](https://mlsanity.com/paper/2512.05000/) (2025)
- [Adaptive Network Combination for Single-Image Reflection Removal: A Domain Generalization Perspective](https://mlsanity.com/paper/2204.01505/) (2022)
- [Seeing Behind Dynamic Occlusions with Event Cameras](https://mlsanity.com/paper/2307.15829/) (2023)
- [Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats](https://mlsanity.com/paper/2410.02764/) (2024)
- [GFRRN: Explore the Gaps in Single Image Reflection Removal](https://mlsanity.com/paper/2602.22695/) (2026)
- [DeclutterNeRF: Generative-Free 3D Scene Recovery for Occlusion Removal](https://mlsanity.com/paper/2504.04679/) (2025)
- [SNNSIR: A Simple Spiking Neural Network for Stereo Image Restoration](https://mlsanity.com/paper/2508.12271/) (2025)

## Abstract (arXiv)

We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions or raindrops, from a short sequence of images captured by a moving camera. Our method leverages the motion differences between the background and the obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. The learning-based layer reconstruction allows us to accommodate potential errors in the flow estimation and brittle assumptions such as brightness consistency. We show that training on synthetically generated data transfers well to real images. Our results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.