# Consensus Learning with Deep Sets for Essential Matrix Estimation

arXiv: https://arxiv.org/abs/2406.17414 · HTML: https://mlsanity.com/paper/2406.17414/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [Graph-Cut RANSAC](https://mlsanity.com/paper/1706.00984/) (2017)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching](https://mlsanity.com/paper/2308.08479/) (2023)
- [Universal Approximation of Functions on Sets](https://mlsanity.com/paper/2107.01959/) (2021)
- [Progressive Correspondence Pruning by Consensus Learning](https://mlsanity.com/paper/2101.00591/) (2021)
- [DeDoDe v2: Analyzing and Improving the DeDoDe Keypoint Detector](https://mlsanity.com/paper/2404.08928/) (2024)
- [Multi-View Optimization of Local Feature Geometry](https://mlsanity.com/paper/2003.08348/) (2020)
- [Learnable Motion Coherence for Correspondence Pruning](https://mlsanity.com/paper/2011.14563/) (2020)
- [Algebraic Characterization of Essential Matrices and Their Averaging in Multiview Settings](https://mlsanity.com/paper/1904.02663/) (2019)
- [Neural Outlier Rejection for Self-Supervised Keypoint Learning](https://mlsanity.com/paper/1912.10615/) (2019)
- [Deep Permutation Equivariant Structure from Motion](https://mlsanity.com/paper/2104.06703/) (2021)
- [BCLNet: Bilateral Consensus Learning for Two-View Correspondence Pruning](https://mlsanity.com/paper/2401.03459/) (2024)
- [MGNet: Learning Correspondences via Multiple Graphs](https://mlsanity.com/paper/2401.04984/) (2024)
- [Optimal Multi-view Correction of Local Affine Frames](https://mlsanity.com/paper/1905.00519/) (2019)

## Built on by (most cited first, 1 of 1)

- [Non-Minimal Sampling and Consensus for Prohibitively Large Datasets](https://mlsanity.com/paper/2604.22518/) (2026)

## Abstract (arXiv)

Robust estimation of the essential matrix, which encodes the relative position and orientation of two cameras, is a fundamental step in structure from motion pipelines. Recent deep-based methods achieved accurate estimation by using complex network architectures that involve graphs, attention layers, and hard pruning steps. Here, we propose a simpler network architecture based on Deep Sets. Given a collection of point matches extracted from two images, our method identifies outlier point matches and models the displacement noise in inlier matches. A weighted DLT module uses these predictions to regress the essential matrix. Our network achieves accurate recovery that is superior to existing networks with significantly more complex architectures.