mlsanityData to 14 Oct 2026

Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares

arXiv 2305.09527 ↗First posted 16 May 2023cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

We propose a differentiable nonlinear least squares framework to account for uncertainty in relative pose estimation from feature correspondences. Specifically, we introduce a symmetric version of the probabilistic normal epipolar constraint, and an approach to estimate the covariance of feature positions by differentiating through the camera pose estimation procedure. We evaluate our approach on synthetic, as well as the KITTI and EuRoC real-world datasets. On the synthetic dataset, we confirm that our learned covariances accurately approximate the true noise distribution. In real world experiments, we find that our approach consistently outperforms state-of-the-art non-probabilistic and probabilistic approaches, regardless of the feature extraction algorithm of choice.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 26 of its 77 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

26 of 77 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  7. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 941
  8. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  9. 2016Direct Sparse Odometrycited by 485
  10. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  11. 2016LIFT: Learned Invariant Feature Transformcited by 306
  12. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 266
14 more
  1. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  2. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 200
  3. 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
  4. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  5. 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 65
  6. 2019Visual-Inertial Mapping with Non-Linear Factor Recoverycited by 50
  7. 2019GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalizationcited by 32
  8. 2019gradSLAM: Automagically differentiable SLAMcited by 27
  9. 2020S2DNet: Learning Accurate Correspondences for Sparse-to-Dense Feature Matchingcited by 23
  10. 2018Geometric Consistency for Self-Supervised End-to-End Visual Odometrycited by 11
  11. 2021Radar Odometry Combining Probabilistic Estimation and Unsupervised Feature Learningcited by 10
  12. 2022The Probabilistic Normal Epipolar Constraint for Frame-To-Frame Rotation Optimization under Uncertain Feature Positionscited by 8
  13. 2020Monocular Rotational Odometry with Incremental Rotation Averaging and Loop Closurecited by 7
  14. 2017Quaternion Based Camera Pose Estimation From Matched Feature Pointscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 2; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024Robust Incremental Structure-from-Motion with Hybrid Featurescited by 31
  2. 2024MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometrycited by 19
  3. 2025AnyCam: Learning to Recover Camera Poses and Intrinsics from Casual Videoscited by 18
  4. 2024MambaVO: Deep Visual Odometry Based on Sequential Matching Refinement and Training Smoothingcited by 5
  5. 2023DAC: Detector-Agnostic Spatial Covariances for Deep Local Featurescited by 3
  6. 2026RaCo: Ranking and Covariance for Practical Learned Keypointscited by 3
  7. 2025BADGR: Bundle Adjustment Diffusion Conditioned by GRadients for Wide-Baseline Floor Plan Reconstructioncited by 2
  8. 2024From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layerscited by 1
  9. 2026ViBA: Implicit Bundle Adjustment with Geometric and Temporal Consistency for Robust Visual Matchingcited by 1
  10. 2026A Pose-only Geometric Constraint for Multi-Camera Pose Adjustmentcited by 1
  11. 2026VOCA: Visual Odometry with Codec Awarenesscited by 1
  12. 2023Residual Learning for Image Point Descriptorscited by 0
2 more
  1. 2026KLTNet: Learning Sparse Feature Tracking for Robust and Accurate Monocular Visual-Inertial Odometrycited by 0
  2. 2026MomentBA: Second-order Spatial Moments for Anisotropic Correspondence Uncertainty in Differentiable Bundle Adjustmentcited by 0