# Manual Labelling Artificially Inflates Deep Learning-Based Segmentation Performance on RGB Images of Closed Canopy: Validation Using TLS

arXiv: https://arxiv.org/abs/2503.14273 · HTML: https://mlsanity.com/paper/2503.14273/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation](https://mlsanity.com/paper/2001.00309/) (2020)
- [Automated forest inventory: analysis of high-density airborne LiDAR point clouds with 3D deep learning](https://mlsanity.com/paper/2312.15084/) (2023)
- [SegmentAnyTree: A sensor and platform agnostic deep learning model for tree segmentation using laser scanning data](https://mlsanity.com/paper/2401.15739/) (2024)
- [Point2Tree(P2T) -- framework for parameter tuning of semantic and instance segmentation used with mobile laser scanning data in coniferous forest](https://mlsanity.com/paper/2305.02651/) (2023)
- [AI applications in forest monitoring need remote sensing benchmark datasets](https://mlsanity.com/paper/2212.09937/) (2022)
- [Large Scale Masked Autoencoding for Reducing Label Requirements on SAR Data](https://mlsanity.com/paper/2310.00826/) (2023)
- [Deep Learning based Automated Forest Health Diagnosis from Aerial Images](https://mlsanity.com/paper/2010.08437/) (2020)
- [Low-Cost Tree Crown Dieback Estimation Using Deep Learning-Based Segmentation](https://mlsanity.com/paper/2409.08171/) (2024)

## Built on by (most cited first, 2 of 2)

- [Benchmarking individual tree segmentation using multispectral airborne laser scanning data: the FGI-EMIT dataset](https://mlsanity.com/paper/2511.00653/) (2025)
- [deadtrees.earth-aerial: A Multi-Resolution Aerial Image Dataset for Tree Cover and Mortality Detection](https://mlsanity.com/paper/2605.19605/) (2026)

## Abstract (arXiv)

Monitoring forest dynamics at an individual tree scale is essential for accurately assessing ecosystem responses to climate change, yet traditional methods relying on field-based forest inventories are labor-intensive and limited in spatial coverage. Advances in remote sensing using drone-acquired RGB imagery combined with deep learning models have promised precise individual tree crown (ITC) segmentation; however, existing methods are frequently validated against human-annotated images, lacking rigorous independent ground truth. In this study, we generate high-fidelity validation labels from co-located Terrestrial Laser Scanning (TLS) data for drone imagery of mixed unmanaged boreal and Mediterranean forests. We evaluate the performance of two widely used deep learning ITC segmentation models - DeepForest (RetinaNet) and Detectree2 (Mask R-CNN) - on these data, and compare to performance on further Mediterranean forest data labelled manually. When validated against TLS-derived ground truth from Mediterranean forests, model performance decreased significantly compared to assessment based on hand-labelled from an ecologically similar site (AP50: 0.094 vs. 0.670). Restricting evaluation to only canopy trees shrank this gap considerably (Canopy AP50: 0.365), although performance was still far lower than on similar hand-labelled data. Models also performed poorly on boreal forest data (AP50: 0.142), although again increasing when evaluated on canopy trees only (Canopy AP50: 0.308). Both models showed very poor localisation accuracy at stricter IoU thresholds, even when restricted to canopy trees (Max AP75: 0.051). Similar results have been observed in studies using aerial LiDAR data, suggesting fundamental limitations in aerial-based segmentation approaches in closed canopy forests.