# Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

arXiv: https://arxiv.org/abs/1905.04132 · HTML: https://mlsanity.com/paper/1905.04132/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2019.

## Built on by (most cited first, 30 of 129)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator](https://mlsanity.com/paper/2102.12145/) (2021)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2203.09418/) (2022)
- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [Learning to Match Features with Seeded Graph Matching Network](https://mlsanity.com/paper/2108.08771/) (2021)
- [PDC-Net+: Enhanced Probabilistic Dense Correspondence Network](https://mlsanity.com/paper/2109.13912/) (2021)
- [GlueStick: Robust Image Matching by Sticking Points and Lines Together](https://mlsanity.com/paper/2304.02008/) (2023)
- [On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation](https://mlsanity.com/paper/2109.00524/) (2021)
- [SC^2-PCR: A Second Order Spatial Compatibility for Efficient and Robust Point Cloud Registration](https://mlsanity.com/paper/2203.14453/) (2022)
- [ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching](https://mlsanity.com/paper/2204.11700/) (2022)
- [Direct-PoseNet: Absolute Pose Regression with Photometric Consistency](https://mlsanity.com/paper/2104.04073/) (2021)
- [CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2111.10502/) (2021)
- [SiLK -- Simple Learned Keypoints](https://mlsanity.com/paper/2304.06194/) (2023)
- [GeoCalib: Learning Single-image Calibration with Geometric Optimization](https://mlsanity.com/paper/2409.06704/) (2024)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [Progressive Correspondence Pruning by Consensus Learning](https://mlsanity.com/paper/2101.00591/) (2021)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering](https://mlsanity.com/paper/2102.11870/) (2021)
- [Map-Relative Pose Regression for Visual Re-Localization](https://mlsanity.com/paper/2404.09884/) (2024)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021)
- [Viewpoint Invariant Dense Matching for Visual Geolocalization](https://mlsanity.com/paper/2109.09827/) (2021)
- [Robust Outlier Rejection for 3D Registration with Variational Bayes](https://mlsanity.com/paper/2304.01514/) (2023)
- [Generalized Differentiable RANSAC](https://mlsanity.com/paper/2212.13185/) (2022)
- [CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data](https://mlsanity.com/paper/2112.09081/) (2021)

## Abstract (arXiv)

We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled fashion that lets us optimize an arbitrary task loss during training, leading to large improvements on classic computer vision tasks. We present two further extensions to NG-RANSAC. Firstly, using the inlier count itself as training signal allows us to train neural guidance in a self-supervised fashion. Secondly, we combine neural guidance with differentiable RANSAC to build neural networks which focus on certain parts of the input data and make the output predictions as good as possible. We evaluate NG-RANSAC on a wide array of computer vision tasks, namely estimation of epipolar geometry, horizon line estimation and camera re-localization. We achieve superior or competitive results compared to state-of-the-art robust estimators, including very recent, learned ones.