# Hierarchical end-to-end autonomous navigation through few-shot waypoint detection

arXiv: https://arxiv.org/abs/2409.14633 · HTML: https://mlsanity.com/paper/2409.14633/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation](https://mlsanity.com/paper/2105.09371/) (2021)
- [Gaussian Prototypical Networks for Few-Shot Learning on Omniglot](https://mlsanity.com/paper/1708.02735/) (2017)
- [Fast and Robust Bio-inspired Teach and Repeat Navigation](https://mlsanity.com/paper/2010.11326/) (2020)
- [Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environments](https://mlsanity.com/paper/2302.04823/) (2023)
- [Agronav: Autonomous Navigation Framework for Agricultural Robots and Vehicles using Semantic Segmentation and Semantic Line Detection](https://mlsanity.com/paper/2304.04333/) (2023)
- [A Driving Intention Prediction Method Based on Hidden Markov Model for Autonomous Driving](https://mlsanity.com/paper/1902.09068/) (2019)
- [Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning](https://mlsanity.com/paper/2101.00186/) (2021)
- [Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars](https://mlsanity.com/paper/2102.00675/) (2021)
- [Learning Real-world Autonomous Navigation by Self-Supervised Environment Synthesis](https://mlsanity.com/paper/2210.04852/) (2022)
- [Indoor Point-to-Point Navigation with Deep Reinforcement Learning and Ultra-wideband](https://mlsanity.com/paper/2011.09241/) (2020)
- [Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm](https://mlsanity.com/paper/2102.00168/) (2021)
- [Real-time Vision-based Navigation for a Robot in an Indoor Environment](https://mlsanity.com/paper/2307.00666/) (2023)
- [One-Shot Learning of Visual Path Navigation for Autonomous Vehicles](https://mlsanity.com/paper/2306.08865/) (2023)

## Built on by (most cited first, 3 of 3)

- [Towards General Urban Monitoring with Vision-Language Models: A Review, Evaluation, and a Research Agenda](https://mlsanity.com/paper/2510.12400/) (2025)
- [MarineFormer: A Spatio-Temporal Attention Model for USV Navigation in Dynamic Marine Environments](https://mlsanity.com/paper/2410.13973/) (2024)
- [One-Cycle Structured Pruning via Stability-Driven Subnetwork Search](https://mlsanity.com/paper/2501.13439/) (2025)

## Abstract (arXiv)

Human navigation is facilitated through the association of actions with landmarks, tapping into our ability to recognize salient features in our environment. Consequently, navigational instructions for humans can be extremely concise, such as short verbal descriptions, indicating a small memory requirement and no reliance on complex and overly accurate navigation tools. Conversely, current autonomous navigation schemes rely on accurate positioning devices and algorithms as well as extensive streams of sensory data collected from the environment. Inspired by this human capability and motivated by the associated technological gap, in this work we propose a hierarchical end-to-end meta-learning scheme that enables a mobile robot to navigate in a previously unknown environment upon presentation of only a few sample images of a set of landmarks along with their corresponding high-level navigation actions. This dramatically simplifies the wayfinding process and enables easy adoption to new environments. For few-shot waypoint detection, we implement a metric-based few-shot learning technique through distribution embedding. Waypoint detection triggers the multi-task low-level maneuver controller module to execute the corresponding high-level navigation action. We demonstrate the effectiveness of the scheme using a small-scale autonomous vehicle on novel indoor navigation tasks in several previously unseen environments.