# From Agile Ground to Aerial Navigation: Learning from Learned Hallucination

arXiv: https://arxiv.org/abs/2108.09793 · HTML: https://mlsanity.com/paper/2108.09793/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2021.

## Built on by (most cited first, 9 of 9)

- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation](https://mlsanity.com/paper/2209.10780/) (2022)
- [VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics](https://mlsanity.com/paper/2203.15983/) (2022)
- [High-Speed Accurate Robot Control using Learned Forward Kinodynamics and Non-linear Least Squares Optimization](https://mlsanity.com/paper/2206.08487/) (2022)
- [Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2409.02383/) (2024)
- [Grounded Curriculum Learning](https://mlsanity.com/paper/2409.19816/) (2024)
- [A Study on Learning Social Robot Navigation with Multimodal Perception](https://mlsanity.com/paper/2309.12568/) (2023)
- [Dyna-LfLH: Learning Agile Navigation in Dynamic Environments from Learned Hallucination](https://mlsanity.com/paper/2403.17231/) (2024)
- [Reinforcement Learning for Adaptive Planner Parameter Tuning: A Perspective on Hierarchical Architecture](https://mlsanity.com/paper/2503.18366/) (2025)

## Abstract (arXiv)

This paper presents a self-supervised Learning from Learned Hallucination (LfLH) method to learn fast and reactive motion planners for ground and aerial robots to navigate through highly constrained environments. The recent Learning from Hallucination (LfH) paradigm for autonomous navigation executes motion plans by random exploration in completely safe obstacle-free spaces, uses hand-crafted hallucination techniques to add imaginary obstacles to the robot's perception, and then learns motion planners to navigate in realistic, highly-constrained, dangerous spaces. However, current hand-crafted hallucination techniques need to be tailored for specific robot types (e.g., a differential drive ground vehicle), and use approximations heavily dependent on certain assumptions (e.g., a short planning horizon). In this work, instead of manually designing hallucination functions, LfLH learns to hallucinate obstacle configurations, where the motion plans from random exploration in open space are optimal, in a self-supervised manner. LfLH is robust to different robot types and does not make assumptions about the planning horizon. Evaluated in both simulated and physical environments with a ground and an aerial robot, LfLH outperforms or performs comparably to previous hallucination approaches, along with sampling- and optimization-based classical methods.