# Survey on Datasets for Perception in Unstructured Outdoor Environments

arXiv: https://arxiv.org/abs/2404.18750 · HTML: https://mlsanity.com/paper/2404.18750/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 30 of 31)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark](https://mlsanity.com/paper/1704.03847/) (2017)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [How do neural networks see depth in single images?](https://mlsanity.com/paper/1905.07005/) (2019)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [Self-Supervised Model Adaptation for Multimodal Semantic Segmentation](https://mlsanity.com/paper/1808.03833/) (2018)
- [The GOOSE Dataset for Perception in Unstructured Environments](https://mlsanity.com/paper/2310.16788/) (2023)
- [ORFD: A Dataset and Benchmark for Off-Road Freespace Detection](https://mlsanity.com/paper/2206.09907/) (2022)
- [EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenes](https://mlsanity.com/paper/2011.04389/) (2020)
- [Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation](https://mlsanity.com/paper/2201.04501/) (2022)
- [TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks](https://mlsanity.com/paper/2402.01913/) (2024)
- [BotanicGarden: A High-Quality Dataset for Robot Navigation in Unstructured Natural Environments](https://mlsanity.com/paper/2306.14137/) (2023)
- [Toward Wheeled Mobility on Vertically Challenging Terrain: Platforms, Datasets, and Algorithms](https://mlsanity.com/paper/2303.00998/) (2023)
- [A Fine-Grained Dataset and its Efficient Semantic Segmentation for Unstructured Driving Scenarios](https://mlsanity.com/paper/2103.13109/) (2021)
- [Instance Segmentation for Autonomous Log Grasping in Forestry Operations](https://mlsanity.com/paper/2203.01902/) (2022)
- [VPAIR -- Aerial Visual Place Recognition and Localization in Large-scale Outdoor Environments](https://mlsanity.com/paper/2205.11567/) (2022)
- [Tree Detection and Diameter Estimation Based on Deep Learning](https://mlsanity.com/paper/2210.17424/) (2022)
- [Tree Species Identification from Bark Images Using Convolutional Neural Networks](https://mlsanity.com/paper/1803.00949/) (2018)
- [Place recognition in gardens by learning visual representations: data set and benchmark analysis](https://mlsanity.com/paper/1906.12151/) (2019)
- [FinnWoodlands Dataset](https://mlsanity.com/paper/2304.00793/) (2023)
- [LiDAR-as-Camera for End-to-End Driving](https://mlsanity.com/paper/2206.15170/) (2022)
- [Training Deep Learning Algorithms on Synthetic Forest Images for Tree Detection](https://mlsanity.com/paper/2210.04104/) (2022)

## Built on by (most cited first, 3 of 3)

- [Excavating in the Wild: The GOOSE-Ex Dataset for Semantic Segmentation](https://mlsanity.com/paper/2409.18788/) (2024)
- [Autonomous Driving in Unstructured Environments: How Far Have We Come?](https://mlsanity.com/paper/2410.07701/) (2024)
- [Diffusion-Based Image Augmentation for Semantic Segmentation in Outdoor Robotics](https://mlsanity.com/paper/2507.00153/) (2025)

## Abstract (arXiv)

Perception is an essential component of pipelines in field robotics. In this survey, we quantitatively compare publicly available datasets available in unstructured outdoor environments. We focus on datasets for common perception tasks in field robotics. Our survey categorizes and compares available research datasets. This survey also reports on relevant dataset characteristics to help practitioners determine which dataset fits best for their own application. We believe more consideration should be taken in choosing compatible annotation policies across the datasets in unstructured outdoor environments.