# Bringing SAM to new heights: Leveraging elevation data for tree crown segmentation from drone imagery

arXiv: https://arxiv.org/abs/2506.04970 · HTML: https://mlsanity.com/paper/2506.04970/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 71 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [SAM on Medical Images: A Comprehensive Study on Three Prompt Modes](https://mlsanity.com/paper/2305.00035/) (2023)
- [Very high resolution canopy height maps from RGB imagery using self-supervised vision transformer and convolutional decoder trained on Aerial Lidar](https://mlsanity.com/paper/2304.07213/) (2023)
- [SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2304.09148/) (2023)
- [UV-SAM: Adapting Segment Anything Model for Urban Village Identification](https://mlsanity.com/paper/2401.08083/) (2024)
- [Automated forest inventory: analysis of high-density airborne LiDAR point clouds with 3D deep learning](https://mlsanity.com/paper/2312.15084/) (2023)
- [ReforesTree: A Dataset for Estimating Tropical Forest Carbon Stock with Deep Learning and Aerial Imagery](https://mlsanity.com/paper/2201.11192/) (2022)
- [Estimating Canopy Height at Scale](https://mlsanity.com/paper/2406.01076/) (2024)
- [OAM-TCD: A globally diverse dataset of high-resolution tree cover maps](https://mlsanity.com/paper/2407.11743/) (2024)
- [Sub-Meter Tree Height Mapping of California using Aerial Images and LiDAR-Informed U-Net Model](https://mlsanity.com/paper/2306.01936/) (2023)
- [Unified Deep Learning Model for Global Prediction of Aboveground Biomass, Canopy Height and Cover from High-Resolution, Multi-Sensor Satellite Imagery](https://mlsanity.com/paper/2408.11234/) (2024)
- [High Resolution Tree Height Mapping of the Amazon Forest using Planet NICFI Images and LiDAR-Informed U-Net Model](https://mlsanity.com/paper/2501.10600/) (2025)
- [Lidar-based Norwegian tree species detection using deep learning](https://mlsanity.com/paper/2311.06066/) (2023)
- [Tree semantic segmentation from aerial image time series](https://mlsanity.com/paper/2407.13102/) (2024)

## Built on by (most cited first, 6 of 6)

- [SelvaMask: Segmenting Trees in Tropical Forests and Beyond](https://mlsanity.com/paper/2602.02426/) (2026)
- [FG-TreeSeg: Flow-Guided Tree Crown Segmentation without Instance Annotations](https://mlsanity.com/paper/2602.00470/) (2026)
- [Estimating Individual Tree Height and Species from UAV Imagery](https://mlsanity.com/paper/2603.23669/) (2026)
- [Understanding Representation Gaps Across Scales in Tropical Tree Species Classification from Drone Imagery](https://mlsanity.com/paper/2604.23019/) (2026)
- [Toward a foundation model for forest point clouds](https://mlsanity.com/paper/2609.24787/) (2026)
- [Latent Commonality Expectation-Maximisation for Box-supervised Tree Crown Instance Segmentation](https://mlsanity.com/paper/2609.26549/) (2026)

## Abstract (arXiv)

Information on trees at the individual level is crucial for monitoring forest ecosystems and planning forest management. Current monitoring methods involve ground measurements, requiring extensive cost, time and labor. Advances in drone remote sensing and computer vision offer great potential for mapping individual trees from aerial imagery at broad-scale. Large pre-trained vision models, such as the Segment Anything Model (SAM), represent a particularly compelling choice given limited labeled data. In this work, we compare methods leveraging SAM for the task of automatic tree crown instance segmentation in high resolution drone imagery in three use cases: 1) boreal plantations, 2) temperate forests and 3) tropical forests. We also study the integration of elevation data into models, in the form of Digital Surface Model (DSM) information, which can readily be obtained at no additional cost from RGB drone imagery. We present BalSAM, a model leveraging SAM and DSM information, which shows potential over other methods, particularly in the context of plantations. We find that methods using SAM out-of-the-box do not outperform a custom Mask R-CNN, even with well-designed prompts. However, efficiently tuning SAM end-to-end and integrating DSM information are both promising avenues for tree crown instance segmentation models.