# KFNet: Learning Temporal Camera Relocalization using Kalman Filtering

arXiv: https://arxiv.org/abs/2003.10629 · HTML: https://mlsanity.com/paper/2003.10629/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2020.

## Built on by (most cited first, 30 of 43)

- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Learning to Match Features with Seeded Graph Matching Network](https://mlsanity.com/paper/2108.08771/) (2021)
- [Neural Refinement for Absolute Pose Regression with Feature Synthesis](https://mlsanity.com/paper/2303.10087/) (2023)
- [CoordiNet: uncertainty-aware pose regressor for reliable vehicle localization](https://mlsanity.com/paper/2103.10796/) (2021)
- [VS-Net: Voting with Segmentation for Visual Localization](https://mlsanity.com/paper/2105.10886/) (2021)
- [Learning Camera Localization via Dense Scene Matching](https://mlsanity.com/paper/2103.16792/) (2021)
- [CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data](https://mlsanity.com/paper/2112.09081/) (2021)
- [Reassessing the Limitations of CNN Methods for Camera Pose Regression](https://mlsanity.com/paper/2108.07260/) (2021)
- [Continual Learning for Image-Based Camera Localization](https://mlsanity.com/paper/2108.09112/) (2021)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [SC-wLS: Towards Interpretable Feed-forward Camera Re-localization](https://mlsanity.com/paper/2210.12748/) (2022)
- [SACReg: Scene-Agnostic Coordinate Regression for Visual Localization](https://mlsanity.com/paper/2307.11702/) (2023)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter](https://mlsanity.com/paper/2407.13185/) (2024)
- [Improving Feature-based Visual Localization by Geometry-Aided Matching](https://mlsanity.com/paper/2211.08712/) (2022)
- [Lazy Visual Localization via Motion Averaging](https://mlsanity.com/paper/2307.09981/) (2023)
- [Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments](https://mlsanity.com/paper/2304.07250/) (2023)
- [Fast and Lightweight Scene Regressor for Camera Relocalization](https://mlsanity.com/paper/2212.01830/) (2022)
- [OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes](https://mlsanity.com/paper/2308.11928/) (2023)
- [Representing 3D sparse map points and lines for camera relocalization](https://mlsanity.com/paper/2402.18011/) (2024)
- [AI-Aided Kalman Filters](https://mlsanity.com/paper/2410.12289/) (2024)
- [Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regression](https://mlsanity.com/paper/2208.00919/) (2022)
- [Robust Monocular Localization of Drones by Adapting Domain Maps to Depth Prediction Inaccuracies](https://mlsanity.com/paper/2210.15559/) (2022)
- [KARNet: Kalman Filter Augmented Recurrent Neural Network for Learning World Models in Autonomous Driving Tasks](https://mlsanity.com/paper/2305.14644/) (2023)
- [Leveraging Neural Radiance Field in Descriptor Synthesis for Keypoints Scene Coordinate Regression](https://mlsanity.com/paper/2403.10297/) (2024)
- [6D Camera Relocalization in Visually Ambiguous Extreme Environments](https://mlsanity.com/paper/2207.06333/) (2022)
- [Enhancing Scene Coordinate Regression with Efficient Keypoint Detection and Sequential Information](https://mlsanity.com/paper/2412.06488/) (2024)
- [LEADER: Learning Reliable Local-to-Global Correspondences for LiDAR Relocalization](https://mlsanity.com/paper/2604.11355/) (2026)
- [3D Surfel Map-Aided Visual Relocalization with Learned Descriptors](https://mlsanity.com/paper/2104.03856/) (2021)

## Abstract (arXiv)

Temporal camera relocalization estimates the pose with respect to each video frame in sequence, as opposed to one-shot relocalization which focuses on a still image. Even though the time dependency has been taken into account, current temporal relocalization methods still generally underperform the state-of-the-art one-shot approaches in terms of accuracy. In this work, we improve the temporal relocalization method by using a network architecture that incorporates Kalman filtering (KFNet) for online camera relocalization. In particular, KFNet extends the scene coordinate regression problem to the time domain in order to recursively establish 2D and 3D correspondences for the pose determination. The network architecture design and the loss formulation are based on Kalman filtering in the context of Bayesian learning. Extensive experiments on multiple relocalization benchmarks demonstrate the high accuracy of KFNet at the top of both one-shot and temporal relocalization approaches. Our codes are released at https://github.com/zlthinker/KFNet.