# What you see is what you get: Experience ranking with deep neural dataset-to-dataset similarity for topological localisation

arXiv: https://arxiv.org/abs/2310.13622 · HTML: https://mlsanity.com/paper/2310.13622/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [Rethinking Visual Geo-localization for Large-Scale Applications](https://mlsanity.com/paper/2204.02287/) (2022)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [MixVPR: Feature Mixing for Visual Place Recognition](https://mlsanity.com/paper/2303.02190/) (2023)
- [Deep Learning Features at Scale for Visual Place Recognition](https://mlsanity.com/paper/1701.05105/) (2017)
- [On the Performance of ConvNet Features for Place Recognition](https://mlsanity.com/paper/1501.04158/) (2015)
- [Mugs: A Multi-Granular Self-Supervised Learning Framework](https://mlsanity.com/paper/2203.14415/) (2022)
- [Single-View Place Recognition under Seasonal Changes](https://mlsanity.com/paper/1808.06516/) (2018)
- [A Bag-of-Prototypes Representation for Dataset-Level Applications](https://mlsanity.com/paper/2303.13251/) (2023)
- [There's No Place Like Home: Visual Teach and Repeat for Emergency Return of Multirotor UAVs During GPS Failure](https://mlsanity.com/paper/1809.05757/) (2018)
- [Visual DNA: Representing and Comparing Images using Distributions of Neuron Activations](https://mlsanity.com/paper/2304.10036/) (2023)
- [Training a Convolutional Neural Network for Appearance-Invariant Place Recognition](https://mlsanity.com/paper/1505.07428/) (2015)
- [Feature Map Filtering: Improving Visual Place Recognition with Convolutional Calibration](https://mlsanity.com/paper/1810.12465/) (2018)

## Built on by (most cited first, 1 of 1)

- [VDNA-PR: Using General Dataset Representations for Robust Sequential Visual Place Recognition](https://mlsanity.com/paper/2403.09025/) (2024)

## Abstract (arXiv)

Recalling the most relevant visual memories for localisation or understanding a priori the likely outcome of localisation effort against a particular visual memory is useful for efficient and robust visual navigation. Solutions to this problem should be divorced from performance appraisal against ground truth - as this is not available at run-time - and should ideally be based on generalisable environmental observations. For this, we propose applying the recently developed Visual DNA as a highly scalable tool for comparing datasets of images - in this work, sequences of map and live experiences. In the case of localisation, important dataset differences impacting performance are modes of appearance change, including weather, lighting, and season. Specifically, for any deep architecture which is used for place recognition by matching feature volumes at a particular layer, we use distribution measures to compare neuron-wise activation statistics between live images and multiple previously recorded past experiences, with a potentially large seasonal (winter/summer) or time of day (day/night) shift. We find that differences in these statistics correlate to performance when localising using a past experience with the same appearance gap. We validate our approach over the Nordland cross-season dataset as well as data from Oxford's University Parks with lighting and mild seasonal change, showing excellent ability of our system to rank actual localisation performance across candidate experiences.