# Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information

arXiv: https://arxiv.org/abs/2407.15593 · HTML: https://mlsanity.com/paper/2407.15593/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [LENS: Localization enhanced by NeRF synthesis](https://mlsanity.com/paper/2110.06558/) (2021)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [DFNet: Enhance Absolute Pose Regression with Direct Feature Matching](https://mlsanity.com/paper/2204.00559/) (2022)
- [Expert Sample Consensus Applied to Camera Re-Localization](https://mlsanity.com/paper/1908.02484/) (2019)
- [A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers](https://mlsanity.com/paper/2207.00254/) (2022)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://mlsanity.com/paper/1702.06521/) (2017)
- [Visual Localization via Few-Shot Scene Region Classification](https://mlsanity.com/paper/2208.06933/) (2022)
- [Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade](https://mlsanity.com/paper/1810.12163/) (2018)
- [Learning Camera Localization via Dense Scene Matching](https://mlsanity.com/paper/2103.16792/) (2021)
- [CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data](https://mlsanity.com/paper/2112.09081/) (2021)
- [Camera Pose Auto-Encoders for Improving Pose Regression](https://mlsanity.com/paper/2207.05530/) (2022)
- [Active Neural Localization](https://mlsanity.com/paper/1801.08214/) (2018)
- [VBR: A Vision Benchmark in Rome](https://mlsanity.com/paper/2404.11322/) (2024)
- [Where to Look Next: Learning Viewpoint Recommendations for Informative Trajectory Planning](https://mlsanity.com/paper/2203.02381/) (2022)
- [Towards Accurate Active Camera Localization](https://mlsanity.com/paper/2012.04263/) (2020)
- [AgriSORT: A Simple Online Real-time Tracking-by-Detection framework for robotics in precision agriculture](https://mlsanity.com/paper/2309.13393/) (2023)
- [Perception-aware Path Planning](https://mlsanity.com/paper/1605.04151/) (2016)
- [Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach](https://mlsanity.com/paper/2310.02650/) (2023)

## Built on by (most cited first, 4 of 4)

- [Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models](https://mlsanity.com/paper/2602.12734/) (2026)
- [ActLoc: Learning to Localize on the Move via Active Viewpoint Selection](https://mlsanity.com/paper/2508.20981/) (2025)
- [Active View Selection for Scene-level Multi-view Crowd Counting and Localization with Limited Labeling Budget](https://mlsanity.com/paper/2509.16684/) (2025)
- [See, Act, Adapt: Active Perception for Unsupervised Cross-Domain Visual Adaptation via Personalized VLM-Guided Agent](https://mlsanity.com/paper/2602.23806/) (2026)

## Abstract (arXiv)

Accurate localization in diverse environments is a fundamental challenge in computer vision and robotics. The task involves determining a sensor's precise position and orientation, typically a camera, within a given space. Traditional localization methods often rely on passive sensing, which may struggle in scenarios with limited features or dynamic environments. In response, this paper explores the domain of active localization, emphasizing the importance of viewpoint selection to enhance localization accuracy. Our contributions involve using a data-driven approach with a simple architecture designed for real-time operation, a self-supervised data training method, and the capability to consistently integrate our map into a planning framework tailored for real-world robotics applications. Our results demonstrate that our method performs better than the existing one, targeting similar problems and generalizing on synthetic and real data. We also release an open-source implementation to benefit the community.