# Vision Transformers, a new approach for high-resolution and large-scale mapping of canopy heights

arXiv: https://arxiv.org/abs/2304.11487 · HTML: https://mlsanity.com/paper/2304.11487/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 91 references that are in the map.
- First posted on arXiv 22 Apr 2023.

## Builds on (most cited first, 21 of 21)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [Incorporating Convolution Designs into Visual Transformers](https://mlsanity.com/paper/2103.11816/) (2021)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Scaling Local Self-Attention for Parameter Efficient Visual Backbones](https://mlsanity.com/paper/2103.12731/) (2021)
- [A high-resolution canopy height model of the Earth](https://mlsanity.com/paper/2204.08322/) (2022)
- [TransAttUnet: Multi-level Attention-guided U-Net with Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2107.05274/) (2021)
- [Country-wide high-resolution vegetation height mapping with Sentinel-2](https://mlsanity.com/paper/1904.13270/) (2019)
- [High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel-2 data with a deep learning approach](https://mlsanity.com/paper/2212.10265/) (2022)
- [Crafting a multi-task CNN for viewpoint estimation](https://mlsanity.com/paper/1609.03894/) (2016)
- [Combining GEDI and Sentinel-2 for wall-to-wall mapping of tall and short crops](https://mlsanity.com/paper/2109.06972/) (2021)

## Built on by (most cited first, 7 of 7)

- [Estimating Canopy Height at Scale](https://mlsanity.com/paper/2406.01076/) (2024)
- [Sub-Meter Tree Height Mapping of California using Aerial Images and LiDAR-Informed U-Net Model](https://mlsanity.com/paper/2306.01936/) (2023)
- [Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation](https://mlsanity.com/paper/2501.19328/) (2025)
- [High Resolution Tree Height Mapping of the Amazon Forest using Planet NICFI Images and LiDAR-Informed U-Net Model](https://mlsanity.com/paper/2501.10600/) (2025)
- [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025)
- [Get Your Embedding Space in Order: Domain-Adaptive Regression for Forest Monitoring](https://mlsanity.com/paper/2405.00514/) (2024)
- [Better Coherence, Better Height: Fusing Physical Models and Deep Learning for Forest Height Estimation from Interferometric SAR Data](https://mlsanity.com/paper/2504.10395/) (2025)

## Abstract (arXiv)

Accurate and timely monitoring of forest canopy heights is critical for assessing forest dynamics, biodiversity, carbon sequestration as well as forest degradation and deforestation. Recent advances in deep learning techniques, coupled with the vast amount of spaceborne remote sensing data offer an unprecedented opportunity to map canopy height at high spatial and temporal resolutions. Current techniques for wall-to-wall canopy height mapping correlate remotely sensed 2D information from optical and radar sensors to the vertical structure of trees using LiDAR measurements. While studies using deep learning algorithms have shown promising performances for the accurate mapping of canopy heights, they have limitations due to the type of architectures and loss functions employed. Moreover, mapping canopy heights over tropical forests remains poorly studied, and the accurate height estimation of tall canopies is a challenge due to signal saturation from optical and radar sensors, persistent cloud covers and sometimes the limited penetration capabilities of LiDARs. Here, we map heights at 10 m resolution across the diverse landscape of Ghana with a new vision transformer (ViT) model optimized concurrently with a classification (discrete) and a regression (continuous) loss function. This model achieves better accuracy than previously used convolutional based approaches (ConvNets) optimized with only a continuous loss function. The ViT model results show that our proposed discrete/continuous loss significantly increases the sensitivity for very tall trees (i.e., > 35m), for which other approaches show saturation effects. The height maps generated by the ViT also have better ground sampling distance and better sensitivity to sparse vegetation in comparison to a convolutional model. Our ViT model has a RMSE of 3.12m in comparison to a reference dataset while the ConvNet model has a RMSE of 4.3m.