# LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization

arXiv: https://arxiv.org/abs/2010.06323 · HTML: https://mlsanity.com/paper/2010.06323/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 28 of 28)

- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [FoundPose: Unseen Object Pose Estimation with Foundation Features](https://mlsanity.com/paper/2311.18809/) (2023)
- [Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image](https://mlsanity.com/paper/2204.04752/) (2022)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [When the Sun Goes Down: Repairing Photometric Losses for All-Day Depth Estimation](https://mlsanity.com/paper/2206.13850/) (2022)
- [The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement](https://mlsanity.com/paper/2404.10438/) (2024)
- [Satellite Image Based Cross-view Localization for Autonomous Vehicle](https://mlsanity.com/paper/2207.13506/) (2022)
- [Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2301.13487/) (2023)
- [DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium](https://mlsanity.com/paper/2304.03560/) (2023)
- [View Consistent Purification for Accurate Cross-View Localization](https://mlsanity.com/paper/2308.08110/) (2023)
- [SAAM: Stealthy Adversarial Attack on Monocular Depth Estimation](https://mlsanity.com/paper/2308.03108/) (2023)
- [APARATE: Adaptive Adversarial Patch for CNN-based Monocular Depth Estimation for Autonomous Navigation](https://mlsanity.com/paper/2303.01351/) (2023)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments](https://mlsanity.com/paper/2304.07250/) (2023)
- [Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2406.05857/) (2024)
- [Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement](https://mlsanity.com/paper/2406.08463/) (2024)
- [SSAP: A Shape-Sensitive Adversarial Patch for Comprehensive Disruption of Monocular Depth Estimation in Autonomous Navigation Applications](https://mlsanity.com/paper/2403.11515/) (2024)
- [Can we make NeRF-based visual localization privacy-preserving?](https://mlsanity.com/paper/2508.18971/) (2025)
- [Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regression](https://mlsanity.com/paper/2208.00919/) (2022)
- [Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimization](https://mlsanity.com/paper/2604.01030/) (2026)
- [Reuse your features: unifying retrieval and feature-metric alignment](https://mlsanity.com/paper/2204.06292/) (2022)
- [Camera Pose Refinement via 3D Gaussian Splatting](https://mlsanity.com/paper/2508.17876/) (2025)
- [Unwarping Screen Content Images via Structure-texture Enhancement Network and Transformation Self-estimation](https://mlsanity.com/paper/2504.15108/) (2025)
- [AlignPose: Generalizable 6D Pose Estimation via Multi-view Feature-metric Alignment](https://mlsanity.com/paper/2512.20538/) (2025)
- [SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2605.28477/) (2026)

## Abstract (arXiv)

We present LM-Reloc -- a novel approach for visual relocalization based on direct image alignment. In contrast to prior works that tackle the problem with a feature-based formulation, the proposed method does not rely on feature matching and RANSAC. Hence, the method can utilize not only corners but any region of the image with gradients. In particular, we propose a loss formulation inspired by the classical Levenberg-Marquardt algorithm to train LM-Net. The learned features significantly improve the robustness of direct image alignment, especially for relocalization across different conditions. To further improve the robustness of LM-Net against large image baselines, we propose a pose estimation network, CorrPoseNet, which regresses the relative pose to bootstrap the direct image alignment. Evaluations on the CARLA and Oxford RobotCar relocalization tracking benchmark show that our approach delivers more accurate results than previous state-of-the-art methods while being comparable in terms of robustness.