# Extreme Relative Pose Estimation for RGB-D Scans via Scene Completion

arXiv: https://arxiv.org/abs/1901.00063 · HTML: https://mlsanity.com/paper/1901.00063/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2018.

## Built on by (most cited first, 22 of 22)

- [PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency](https://mlsanity.com/paper/2103.05465/) (2021)
- [A comprehensive survey on point cloud registration](https://mlsanity.com/paper/2103.02690/) (2021)
- [Extreme Rotation Estimation using Dense Correlation Volumes](https://mlsanity.com/paper/2104.13530/) (2021)
- [The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs](https://mlsanity.com/paper/2208.08988/) (2022)
- [Virtual Correspondence: Humans as a Cue for Extreme-View Geometry](https://mlsanity.com/paper/2206.08365/) (2022)
- [LiDAR-based 4D Occupancy Completion and Forecasting](https://mlsanity.com/paper/2310.11239/) (2023)
- [Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation](https://mlsanity.com/paper/2303.11329/) (2023)
- [PuzzleFusion: Unleashing the Power of Diffusion Models for Spatial Puzzle Solving](https://mlsanity.com/paper/2211.13785/) (2022)
- [Environment Predictive Coding for Embodied Agents](https://mlsanity.com/paper/2102.02337/) (2021)
- [FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation](https://mlsanity.com/paper/2403.03221/) (2024)
- [Scene-agnostic Pose Regression for Visual Localization](https://mlsanity.com/paper/2503.19543/) (2025)
- [Extreme Rotation Estimation in the Wild](https://mlsanity.com/paper/2411.07096/) (2024)
- [Emergent Extreme-View Geometry in 3D Foundation Models](https://mlsanity.com/paper/2511.22686/) (2025)
- [Consistent Two-Flow Network for Tele-Registration of Point Clouds](https://mlsanity.com/paper/2106.00329/) (2021)
- [Visual Correspondence Hallucination](https://mlsanity.com/paper/2106.09711/) (2021)
- [Floorplan Restoration by Structure Hallucinating Transformer Cascades](https://mlsanity.com/paper/2206.00645/) (2022)
- [HM3D-ABO: A Photo-realistic Dataset for Object-centric Multi-view 3D Reconstruction](https://mlsanity.com/paper/2206.12356/) (2022)
- [SALVe: Semantic Alignment Verification for Floorplan Reconstruction from Sparse Panoramas](https://mlsanity.com/paper/2406.19390/) (2024)
- [MGPC: Multimodal Network for Generalizable Point Cloud Completion With Modality Dropout and Progressive Decoding](https://mlsanity.com/paper/2601.03660/) (2026)
- [PoseCrafter: Extreme Pose Estimation with Hybrid Video Synthesis](https://mlsanity.com/paper/2510.19527/) (2025)
- [Geometry Depth Consistency in RGBD Relative Pose Estimation](https://mlsanity.com/paper/2401.00639/) (2024)
- [F2M-Reg: Unsupervised RGB-D Point Cloud Registration with Frame-to-Model Optimization](https://mlsanity.com/paper/2405.00507/) (2024)

## Abstract (arXiv)

Estimating the relative rigid pose between two RGB-D scans of the same underlying environment is a fundamental problem in computer vision, robotics, and computer graphics. Most existing approaches allow only limited maximum relative pose changes since they require considerable overlap between the input scans. We introduce a novel deep neural network that extends the scope to extreme relative poses, with little or even no overlap between the input scans. The key idea is to infer more complete scene information about the underlying environment and match on the completed scans. In particular, instead of only performing scene completion from each individual scan, our approach alternates between relative pose estimation and scene completion. This allows us to perform scene completion by utilizing information from both input scans at late iterations, resulting in better results for both scene completion and relative pose estimation. Experimental results on benchmark datasets show that our approach leads to considerable improvements over state-of-the-art approaches for relative pose estimation. In particular, our approach provides encouraging relative pose estimates even between non-overlapping scans.