# FindView: Precise Target View Localization Task for Look Around Agents

arXiv: https://arxiv.org/abs/2303.09054 · HTML: https://mlsanity.com/paper/2303.09054/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [ViNG: Learning Open-World Navigation with Visual Goals](https://mlsanity.com/paper/2012.09812/) (2020)
- [No RL, No Simulation: Learning to Navigate without Navigating](https://mlsanity.com/paper/2110.09470/) (2021)
- [VTNet: Visual Transformer Network for Object Goal Navigation](https://mlsanity.com/paper/2105.09447/) (2021)
- [Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks](https://mlsanity.com/paper/1709.00507/) (2017)
- [A Memory Network Approach for Story-based Temporal Summarization of 360deg Videos](https://mlsanity.com/paper/1805.02838/) (2018)
- [360-Indoor: Towards Learning Real-World Objects in 360deg Indoor Equirectangular Images](https://mlsanity.com/paper/1910.01712/) (2019)
- [Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization](https://mlsanity.com/paper/1908.06387/) (2019)
- [Pano2Vid: Automatic Cinematography for Watching 360circ Videos](https://mlsanity.com/paper/1612.02335/) (2016)
- [Deep 360 Pilot: Learning a Deep Agent for Piloting through 360deg Sports Video](https://mlsanity.com/paper/1705.01759/) (2017)
- [Active Neural Localization](https://mlsanity.com/paper/1801.08214/) (2018)
- [Emergence of Exploratory Look-Around Behaviors through Active Observation Completion](https://mlsanity.com/paper/1906.11407/) (2019)
- [Making 360circ Video Watchable in 2D: Learning Videography for Click Free Viewing](https://mlsanity.com/paper/1703.00495/) (2017)
- [Sidekick Policy Learning for Active Visual Exploration](https://mlsanity.com/paper/1807.11010/) (2018)
- [Global Pose Estimation with an Attention-based Recurrent Network](https://mlsanity.com/paper/1802.06857/) (2018)
- [Semantic-driven Generation of Hyperlapse from 360^circ Video](https://mlsanity.com/paper/1703.10798/) (2017)

## Abstract (arXiv)

With the increase in demands for service robots and automated inspection, agents need to localize in its surrounding environment to achieve more natural communication with humans by shared contexts. In this work, we propose a novel but straightforward task of precise target view localization for look around agents called the FindView task. This task imitates the movements of PTZ cameras or user interfaces for 360 degree mediums, where the observer must "look around" to find a view that exactly matches the target. To solve this task, we introduce a rule-based agent that heuristically finds the optimal view and a policy learning agent that employs reinforcement learning to learn by interacting with the 360 degree scene. Through extensive evaluations and benchmarks, we conclude that learned methods have many advantages, in particular precise localization that is robust to corruption and can be easily deployed in novel scenes.