# NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge

arXiv: https://arxiv.org/abs/2103.11470 · HTML: https://mlsanity.com/paper/2103.11470/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 119 references that are in the map.
- First posted on arXiv 21 Mar 2021.

## Builds on (most cited first, 30 of 38)

- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [MobileDets: Searching for Object Detection Architectures for Mobile Accelerators](https://mlsanity.com/paper/2004.14525/) (2020)
- [Tightly Coupled 3D Lidar Inertial Odometry and Mapping](https://mlsanity.com/paper/1904.06993/) (2019)
- [DOOR-SLAM: Distributed, Online, and Outlier Resilient SLAM for Robotic Teams](https://mlsanity.com/paper/1909.12198/) (2019)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [Autonomous Spot: Long-Range Autonomous Exploration of Extreme Environments with Legged Locomotion](https://mlsanity.com/paper/2010.09259/) (2020)
- [Bayesian Learning-Based Adaptive Control for Safety Critical Systems](https://mlsanity.com/paper/1910.02325/) (2019)
- [LAMP: Large-Scale Autonomous Mapping and Positioning for Exploration of Perceptually-Degraded Subterranean Environments](https://mlsanity.com/paper/2003.01744/) (2020)
- [LOCUS: A Multi-Sensor Lidar-Centric Solution for High-Precision Odometry and 3D Mapping in Real-Time](https://mlsanity.com/paper/2012.14447/) (2020)
- [Deep Learning Tubes for Tube MPC](https://mlsanity.com/paper/2002.01587/) (2020)
- [Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources](https://mlsanity.com/paper/2002.02921/) (2020)
- [PLGRIM: Hierarchical Value Learning for Large-scale Exploration in Unknown Environments](https://mlsanity.com/paper/2102.05633/) (2021)
- [Distributed Mapping with Privacy and Communication Constraints: Lightweight Algorithms and Object-based Models](https://mlsanity.com/paper/1702.03435/) (2017)
- [DARE-SLAM: Degeneracy-Aware and Resilient Loop Closing in Perceptually-Degraded Environments](https://mlsanity.com/paper/2102.05117/) (2021)
- [Mine Tunnel Exploration using Multiple Quadrupedal Robots](https://mlsanity.com/paper/1909.09662/) (2019)
- [Towards Resilient Autonomous Navigation of Drones](https://mlsanity.com/paper/2008.09679/) (2020)
- [Confidence-rich grid mapping](https://mlsanity.com/paper/2006.15754/) (2020)
- [Bi-directional Value Learning for Risk-aware Planning Under Uncertainty: Extended Version](https://mlsanity.com/paper/1902.05698/) (2019)
- [Where to Map? Iterative Rover-Copter Path Planning for Mars Exploration](https://mlsanity.com/paper/2008.07157/) (2020)
- [LION: Lidar-Inertial Observability-Aware Navigator for Vision-Denied Environments](https://mlsanity.com/paper/2102.03443/) (2021)
- [On the Covariance of ICP-based Scan-matching Techniques](https://mlsanity.com/paper/1410.7632/) (2014)
- [Contact Inertial Odometry: Collisions are your Friends](https://mlsanity.com/paper/1909.00079/) (2019)
- [Virtual Surfaces and Attitude Aware Planning and Behaviours for Negative Obstacle Navigation](https://mlsanity.com/paper/2010.16018/) (2020)
- [Path Planning and Navigation Inside Off-World Lava Tubes and Caves](https://mlsanity.com/paper/1803.02818/) (2018)

## Built on by (most cited first, 30 of 32)

- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [GNM: A General Navigation Model to Drive Any Robot](https://mlsanity.com/paper/2210.03370/) (2022)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems](https://mlsanity.com/paper/2106.14386/) (2021)
- [DarkGS: Learning Neural Illumination and 3D Gaussians Relighting for Robotic Exploration in the Dark](https://mlsanity.com/paper/2403.10814/) (2024)
- [Probabilistic Volumetric Fusion for Dense Monocular SLAM](https://mlsanity.com/paper/2210.01276/) (2022)
- [Learning Risk-Aware Costmaps via Inverse Reinforcement Learning for Off-Road Navigation](https://mlsanity.com/paper/2302.00134/) (2023)
- [DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization](https://mlsanity.com/paper/2212.06331/) (2022)
- [Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results](https://mlsanity.com/paper/2212.06759/) (2022)
- [Learning Risk-aware Costmaps for Traversability in Challenging Environments](https://mlsanity.com/paper/2107.11722/) (2021)
- [Air-Ground Collaboration for Language-Specified Missions in Unknown Environments](https://mlsanity.com/paper/2505.09108/) (2025)
- [Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data](https://mlsanity.com/paper/2210.14721/) (2022)
- [UAVs Beneath the Surface: Cooperative Autonomy for Subterranean Search and Rescue in DARPA SubT](https://mlsanity.com/paper/2206.08185/) (2022)
- [FRAME: Fast and Robust Autonomous 3D point cloud Map-merging for Egocentric multi-robot exploration](https://mlsanity.com/paper/2301.09213/) (2023)
- [RecNet: An Invertible Point Cloud Encoding through Range Image Embeddings for Multi-Robot Map Sharing and Reconstruction](https://mlsanity.com/paper/2402.02192/) (2024)
- [PhotoReg: Photometrically Registering 3D Gaussian Splatting Models](https://mlsanity.com/paper/2410.05044/) (2024)
- [STAGE: Scalable and Traversability-Aware Graph based Exploration Planner for Dynamically Varying Environments](https://mlsanity.com/paper/2402.02566/) (2024)
- [SayComply: Grounding Field Robotic Tasks in Operational Compliance through Retrieval-Based Language Models](https://mlsanity.com/paper/2411.11323/) (2024)
- [Deploying Foundation Model-Enabled Air and Ground Robots in the Field: Challenges and Opportunities](https://mlsanity.com/paper/2505.09477/) (2025)
- [WildOS: Open-Vocabulary Object Search in the Wild](https://mlsanity.com/paper/2602.19308/) (2026)
- [Olympus: A Jumping Quadruped for Planetary Exploration Utilizing Reinforcement Learning for In-Flight Attitude Control](https://mlsanity.com/paper/2503.03574/) (2025)
- [Copiloting Autonomous Multi-Robot Missions: A Game-inspired Supervisory Control Interface](https://mlsanity.com/paper/2204.06647/) (2022)
- [Early Recall, Late Precision: Multi-Robot Semantic Object Mapping under Operational Constraints in Perceptually-Degraded Environments](https://mlsanity.com/paper/2206.10062/) (2022)
- [Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments](https://mlsanity.com/paper/2207.13791/) (2022)
- [Risk-aware Meta-level Decision Making for Exploration Under Uncertainty](https://mlsanity.com/paper/2209.05580/) (2022)
- [Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities](https://mlsanity.com/paper/2210.09678/) (2022)
- [Multimodal Dataset from Harsh Sub-Terranean Environment with Aerosol Particles for Frontier Exploration](https://mlsanity.com/paper/2304.14520/) (2023)
- [Agent Mars: Multi-Agent Simulation for Multi-Planetary Life Exploration and Settlement](https://mlsanity.com/paper/2602.13291/) (2026)
- [SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty](https://mlsanity.com/paper/2603.05111/) (2026)
- [Staircase Localization for Autonomous Exploration in Urban Environments](https://mlsanity.com/paper/2403.17330/) (2024)

## Abstract (arXiv)

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR's demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including: (i) geometric and semantic environment mapping; (ii) a multi-modal positioning system; (iii) traversability analysis and local planning; (iv) global motion planning and exploration behavior; (i) risk-aware mission planning; (vi) networking and decentralized reasoning; and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g. wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition.