# Generalized Differentiable RANSAC

arXiv: https://arxiv.org/abs/2212.13185 · HTML: https://mlsanity.com/paper/2212.13185/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 81 references that are in the map.
- First posted on arXiv 26 Dec 2022.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [Understanding Gradient Clipping in Private SGD: A Geometric Perspective](https://mlsanity.com/paper/2006.15429/) (2020)
- [Graph-Cut RANSAC](https://mlsanity.com/paper/1706.00984/) (2017)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [Stochastic Optimization of Sorting Networks via Continuous Relaxations](https://mlsanity.com/paper/1903.08850/) (2019)
- [Progressive Correspondence Pruning by Consensus Learning](https://mlsanity.com/paper/2101.00591/) (2021)
- [MODS: Fast and Robust Method for Two-View Matching](https://mlsanity.com/paper/1503.02619/) (2015)
- [VSAC: Efficient and Accurate Estimator for H and F](https://mlsanity.com/paper/2106.10240/) (2021)
- [Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution](https://mlsanity.com/paper/1911.10036/) (2019)
- [Learning Surrogates via Deep Embedding](https://mlsanity.com/paper/2007.00799/) (2020)
- [ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning](https://mlsanity.com/paper/1907.02545/) (2019)
- [NeFSAC: Neurally Filtered Minimal Samples](https://mlsanity.com/paper/2207.07872/) (2022)
- [FEDS -- Filtered Edit Distance Surrogate](https://mlsanity.com/paper/2103.04635/) (2021)

## Built on by (most cited first, 30 of 30)

- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion](https://mlsanity.com/paper/2409.19152/) (2024)
- [PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2410.22128/) (2024)
- [GeoCalib: Learning Single-image Calibration with Geometric Optimization](https://mlsanity.com/paper/2409.06704/) (2024)
- [Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs](https://mlsanity.com/paper/2312.07246/) (2023)
- [Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://mlsanity.com/paper/2404.06337/) (2024)
- [MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion](https://mlsanity.com/paper/2504.20040/) (2025)
- [Emergent Outlier View Rejection in Visual Geometry Grounded Transformers](https://mlsanity.com/paper/2512.04012/) (2025)
- [Relative Pose Estimation through Affine Corrections of Monocular Depth Priors](https://mlsanity.com/paper/2501.05446/) (2025)
- [Learning to Make Keypoints Sub-Pixel Accurate](https://mlsanity.com/paper/2407.11668/) (2024)
- [DaD: Distilled Reinforcement Learning for Diverse Keypoint Detection](https://mlsanity.com/paper/2503.07347/) (2025)
- [TurboReg: TurboClique for Robust and Efficient Point Cloud Registration](https://mlsanity.com/paper/2507.01439/) (2025)
- [FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation](https://mlsanity.com/paper/2403.03221/) (2024)
- [HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioning](https://mlsanity.com/paper/2407.15844/) (2024)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [SupeRANSAC: One RANSAC to Rule Them All](https://mlsanity.com/paper/2506.04803/) (2025)
- [Image Matching Filtering and Refinement by Planes and Beyond](https://mlsanity.com/paper/2411.09484/) (2024)
- [Unsupervised Point Cloud Registration with Self-Distillation](https://mlsanity.com/paper/2409.07558/) (2024)
- [MinBackProp -- Backpropagating through Minimal Solvers](https://mlsanity.com/paper/2404.17993/) (2024)
- [Efficient and Distributed Large-Scale 3D Map Registration using Tomographic Features](https://mlsanity.com/paper/2406.19461/) (2024)
- [Monte Carlo Diffusion for Generalizable Learning-Based RANSAC](https://mlsanity.com/paper/2503.09410/) (2025)
- [KVN: Keypoints Voting Network with Differentiable RANSAC for Stereo Pose Estimation](https://mlsanity.com/paper/2307.11543/) (2023)
- [Global Point Cloud Registration Network for Large Transformations](https://mlsanity.com/paper/2403.18040/) (2024)
- [A Birotation Solution for Relative Pose Problems](https://mlsanity.com/paper/2505.02025/) (2025)
- [Event-driven Robust Fitting on Neuromorphic Hardware](https://mlsanity.com/paper/2508.09466/) (2025)
- [End2Reg: Learning Task-Specific Segmentation for Markerless Registration in Spine Surgery](https://mlsanity.com/paper/2512.13402/) (2025)
- [Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing](https://mlsanity.com/paper/2607.09143/) (2026)
- [Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives](https://mlsanity.com/paper/2608.11093/) (2026)
- [DiffSAC: Diffusion-guided Sampling for Consensus-based Robust Estimation](https://mlsanity.com/paper/2608.30603/) (2026)
- [BLASt3R: Bundle Adjustment of Any Image Set with Multi-View Matching and Monocular Priors](https://mlsanity.com/paper/2609.05210/) (2026)

## Abstract (arXiv)

We propose nabla-RANSAC, a generalized differentiable RANSAC that allows learning the entire randomized robust estimation pipeline. The proposed approach enables the use of relaxation techniques for estimating the gradients in the sampling distribution, which are then propagated through a differentiable solver. The trainable quality function marginalizes over the scores from all the models estimated within nabla-RANSAC to guide the network learning accurate and useful inlier probabilities or to train feature detection and matching networks. Our method directly maximizes the probability of drawing a good hypothesis, allowing us to learn better sampling distributions. We test nabla-RANSAC on various real-world scenarios on fundamental and essential matrix estimation, and 3D point cloud registration, outdoors and indoors, with handcrafted and learning-based features. It is superior to the state-of-the-art in terms of accuracy while running at a similar speed to its less accurate alternatives. The code and trained models are available at https://github.com/weitong8591/differentiableransac.