# 3DoF Localization from a Single Image and an Object Map: the Flatlandia Problem and Dataset

arXiv: https://arxiv.org/abs/2304.06373 · HTML: https://mlsanity.com/paper/2304.06373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Back to MLP: A Simple Baseline for Human Motion Prediction](https://mlsanity.com/paper/2207.01567/) (2022)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [OrienterNet: Visual Localization in 2D Public Maps with Neural Matching](https://mlsanity.com/paper/2304.02009/) (2023)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [To Learn or Not to Learn: Visual Localization from Essential Matrices](https://mlsanity.com/paper/1908.01293/) (2019)
- [Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark](https://mlsanity.com/paper/2205.15761/) (2022)
- [You Are Here: Geolocation by Embedding Maps and Images](https://mlsanity.com/paper/1911.08797/) (2019)
- [Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization](https://mlsanity.com/paper/1908.06141/) (2019)
- [Improved Visual Relocalization by Discovering Anchor Points](https://mlsanity.com/paper/1811.04370/) (2018)
- [CrowdDriven: A New Challenging Dataset for Outdoor Visual Localization](https://mlsanity.com/paper/2109.04527/) (2021)
- [3D-Aware Ellipse Prediction for Object-Based Camera Pose Estimation](https://mlsanity.com/paper/2105.11494/) (2021)
- [Object-Guided Day-Night Visual Localization in Urban Scenes](https://mlsanity.com/paper/2202.04445/) (2022)

## Abstract (arXiv)

Efficient visual localization is crucial to many applications, such as large-scale deployment of autonomous agents and augmented reality. Traditional visual localization, while achieving remarkable accuracy, relies on extensive 3D models of the scene or large collections of geolocalized images, which are often inefficient to store and to scale to novel environments. In contrast, humans orient themselves using very abstract 2D maps, using the location of clearly identifiable landmarks. Drawing on this and on the success of recent works that explored localization on 2D abstract maps, we propose Flatlandia, a novel visual localization challenge. With Flatlandia, we investigate whether it is possible to localize a visual query by comparing the layout of its common objects detected against the known spatial layout of objects in the map. We formalize the challenge as two tasks at different levels of accuracy to investigate the problem and its possible limitations; for each, we propose initial baseline models and compare them against state-of-the-art 6DoF and 3DoF methods. Code and dataset are publicly available at github.com/IIT-PAVIS/Flatlandia.