# GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization

arXiv: https://arxiv.org/abs/1904.11932 · HTML: https://mlsanity.com/paper/1904.11932/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2019.

## Built on by (most cited first, 30 of 32)

- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering](https://mlsanity.com/paper/2104.00633/) (2021)
- [CROSSFIRE: Camera Relocalization On Self-Supervised Features from an Implicit Representation](https://mlsanity.com/paper/2303.04869/) (2023)
- [RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization](https://mlsanity.com/paper/2203.12870/) (2022)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [Unsupervised Monocular Depth Estimation in Highly Complex Environments](https://mlsanity.com/paper/2107.13137/) (2021)
- [When the Sun Goes Down: Repairing Photometric Losses for All-Day Depth Estimation](https://mlsanity.com/paper/2206.13850/) (2022)
- [The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement](https://mlsanity.com/paper/2404.10438/) (2024)
- [Probabilistic Visual Place Recognition for Hierarchical Localization](https://mlsanity.com/paper/2105.03091/) (2021)
- [Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimation](https://mlsanity.com/paper/2103.07153/) (2021)
- [DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium](https://mlsanity.com/paper/2304.03560/) (2023)
- [Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares](https://mlsanity.com/paper/2305.09527/) (2023)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [Object-Augmented RGB-D SLAM for Wide-Disparity Relocalisation](https://mlsanity.com/paper/2108.02522/) (2021)
- [Improving Worst Case Visual Localization Coverage via Place-specific Sub-selection in Multi-camera Systems](https://mlsanity.com/paper/2206.13883/) (2022)
- [4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions](https://mlsanity.com/paper/2301.01147/) (2022)
- [Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement](https://mlsanity.com/paper/2406.08463/) (2024)
- [Tight Integration of Feature-based Relocalization in Monocular Direct Visual Odometry](https://mlsanity.com/paper/2102.01191/) (2021)
- [Benchmarking Visual Localization for Autonomous Navigation](https://mlsanity.com/paper/2203.13048/) (2022)
- [Rover Relocalization for Mars Sample Return by Virtual Template Synthesis and Matching](https://mlsanity.com/paper/2103.03395/) (2021)
- [ImPosing: Implicit Pose Encoding for Efficient Visual Localization](https://mlsanity.com/paper/2205.02638/) (2022)
- [Can we make NeRF-based visual localization privacy-preserving?](https://mlsanity.com/paper/2508.18971/) (2025)
- [TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation](https://mlsanity.com/paper/2303.13477/) (2023)
- [Self-Supervised Steering Angle Prediction for Vehicle Control Using Visual Odometry](https://mlsanity.com/paper/2103.11204/) (2021)
- [Semantic Image Alignment for Vehicle Localization](https://mlsanity.com/paper/2110.04162/) (2021)
- [A high-precision self-supervised monocular visual odometry in foggy weather based on robust cycled generative adversarial networks and multi-task learning aided depth estimation](https://mlsanity.com/paper/2203.04812/) (2022)
- [Reuse your features: unifying retrieval and feature-metric alignment](https://mlsanity.com/paper/2204.06292/) (2022)
- [Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency](https://mlsanity.com/paper/2402.09237/) (2024)

## Abstract (arXiv)

Direct SLAM methods have shown exceptional performance on odometry tasks. However, they are susceptible to dynamic lighting and weather changes while also suffering from a bad initialization on large baselines. To overcome this, we propose GN-Net: a network optimized with the novel Gauss-Newton loss for training weather invariant deep features, tailored for direct image alignment. Our network can be trained with pixel correspondences between images taken from different sequences. Experiments on both simulated and real-world datasets demonstrate that our approach is more robust against bad initialization, variations in day-time, and weather changes thereby outperforming state-of-the-art direct and indirect methods. Furthermore, we release an evaluation benchmark for relocalization tracking against different types of weather. Our benchmark is available at https://vision.in.tum.de/gn-net.