# Ground material classification for UAV-based photogrammetric 3D data A 2D-3D Hybrid Approach

arXiv: https://arxiv.org/abs/2109.12221 · HTML: https://mlsanity.com/paper/2109.12221/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 5 of 5)

- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1606.02147/) (2016)
- [3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation](https://mlsanity.com/paper/1803.10409/) (2018)
- [Fully Automated Photogrammetric Data Segmentation and Object Information Extraction Approach for Creating Simulation Terrain](https://mlsanity.com/paper/2008.03697/) (2020)
- [Generating synthetic photogrammetric data for training deep learning based 3D point cloud segmentation models](https://mlsanity.com/paper/2008.09647/) (2020)

## Built on by (most cited first, 3 of 3)

- [STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset](https://mlsanity.com/paper/2203.09065/) (2022)
- [Open-Vocabulary High-Resolution 3D (OVHR3D) Data Segmentation and Annotation Framework](https://mlsanity.com/paper/2412.06268/) (2024)
- [IDU: Incremental Dynamic Update of Existing 3D Virtual Environments with New Imagery Data](https://mlsanity.com/paper/2508.17579/) (2025)

## Abstract (arXiv)

In recent years, photogrammetry has been widely used in many areas to create photorealistic 3D virtual data representing the physical environment. The innovation of small unmanned aerial vehicles (sUAVs) has provided additional high-resolution imaging capabilities with low cost for mapping a relatively large area of interest. These cutting-edge technologies have caught the US Army and Navy's attention for the purpose of rapid 3D battlefield reconstruction, virtual training, and simulations. Our previous works have demonstrated the importance of information extraction from the derived photogrammetric data to create semantic-rich virtual environments (Chen et al., 2019). For example, an increase of simulation realism and fidelity was achieved by segmenting and replacing photogrammetric trees with game-ready tree models. In this work, we further investigated the semantic information extraction problem and focused on the ground material segmentation and object detection tasks. The main innovation of this work was that we leveraged both the original 2D images and the derived 3D photogrammetric data to overcome the challenges faced when using each individual data source. For ground material segmentation, we utilized an existing convolutional neural network architecture (i.e., 3DMV) which was originally designed for segmenting RGB-D sensed indoor data. We improved its performance for outdoor photogrammetric data by introducing a depth pooling layer in the architecture to take into consideration the distance between the source images and the reconstructed terrain model. To test the performance of our improved 3DMV, a ground truth ground material database was created using data from the One World Terrain (OWT) data repository. Finally, a workflow for importing the segmented ground materials into a virtual simulation scene was introduced, and visual results are reported in this paper.