# FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoring

arXiv: https://arxiv.org/abs/2312.10114 · HTML: https://mlsanity.com/paper/2312.10114/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 67 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [Functional Map of the World](https://mlsanity.com/paper/1711.07846/) (2017)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection](https://mlsanity.com/paper/2106.00666/) (2021)
- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023)
- [SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion](https://mlsanity.com/paper/1906.07789/) (2019)
- [BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval](https://mlsanity.com/paper/2105.07921/) (2021)
- [SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning](https://mlsanity.com/paper/2111.04724/) (2021)
- [FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery](https://mlsanity.com/paper/2310.13336/) (2023)
- [ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery](https://mlsanity.com/paper/2011.05479/) (2020)
- [Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark](https://mlsanity.com/paper/2112.00570/) (2021)
- [FOR-instance: a UAV laser scanning benchmark dataset for semantic and instance segmentation of individual trees](https://mlsanity.com/paper/2309.01279/) (2023)
- [A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning](https://mlsanity.com/paper/2204.00951/) (2022)
- [ReforesTree: A Dataset for Estimating Tropical Forest Carbon Stock with Deep Learning and Aerial Imagery](https://mlsanity.com/paper/2201.11192/) (2022)
- [Benchmarking and scaling of deep learning models for land cover image classification](https://mlsanity.com/paper/2111.09451/) (2021)

## Built on by (most cited first, 30 of 31)

- [On the Foundations of Earth and Climate Foundation Models](https://mlsanity.com/paper/2405.04285/) (2024)
- [GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI](https://mlsanity.com/paper/2511.15658/) (2025)
- [Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment](https://mlsanity.com/paper/2402.09816/) (2024)
- [No One Knows the State of the Art in Geospatial Foundation Models](https://mlsanity.com/paper/2605.12678/) (2026)
- [GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data](https://mlsanity.com/paper/2509.26016/) (2025)
- [MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training](https://mlsanity.com/paper/2602.06285/) (2026)
- [FLAIR-HUB: Large-scale Multimodal Dataset for Land Cover and Crop Mapping](https://mlsanity.com/paper/2506.07080/) (2025)
- [Ice-FMBench: A Foundation Model Benchmark for Sea Ice Type Segmentation](https://mlsanity.com/paper/2503.22516/) (2025)
- [SelvaBox: A high-resolution dataset for tropical tree crown detection](https://mlsanity.com/paper/2507.00170/) (2025)
- [UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation](https://mlsanity.com/paper/2606.23503/) (2026)
- [A Genealogy of Foundation Models in Remote Sensing](https://mlsanity.com/paper/2504.17177/) (2025)
- [GlobalGeoTree: A Multi-Granular Vision-Language Dataset for Global Tree Species Classification](https://mlsanity.com/paper/2505.12513/) (2025)
- [GreenHyperSpectra: A multi-source hyperspectral dataset for global vegetation trait prediction](https://mlsanity.com/paper/2507.06806/) (2025)
- [Any Model, Any Place, Any Time: Get Remote Sensing Foundation Model Embeddings On Demand](https://mlsanity.com/paper/2602.23678/) (2026)
- [Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2510.22964/) (2025)
- [UAV (Unmanned Aerial Vehicles): Diverse Applications of UAV Datasets in Segmentation, Classification, Detection, and Tracking](https://mlsanity.com/paper/2409.03245/) (2024)
- [MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Data](https://mlsanity.com/paper/2508.10894/) (2025)
- [HighFM: Towards a Foundation Model for Learning Representations from High-Frequency Earth Observation Data](https://mlsanity.com/paper/2604.04306/) (2026)
- [Agentic AI for Remote Sensing: Technical Challenges and Research Directions](https://mlsanity.com/paper/2604.24919/) (2026)
- [Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challenges](https://mlsanity.com/paper/2508.06832/) (2025)
- [SegAssess: Panoramic quality mapping for robust and transferable unsupervised segmentation assessment](https://mlsanity.com/paper/2509.01183/) (2025)
- [Local Precise Refinement: A Dual-Gated Mixture-of-Experts for Enhancing Foundation Model Generalization against Spectral Shifts](https://mlsanity.com/paper/2603.13352/) (2026)
- [Emerging Flexible Designs for Geospatial Multimodal Foundation Models](https://mlsanity.com/paper/2606.12595/) (2026)
- [Multi-Modal, Multi-Temporal Geospatial Foundation Models Improve Global Above-Ground Biomass Estimation](https://mlsanity.com/paper/2608.04792/) (2026)
- [Bayesian Fusion of Active Contour Models and ConvNet Priors for Standing Dead Tree Segmentation](https://mlsanity.com/paper/2410.07421/) (2024)
- [Atmos-Bench: 3D Atmospheric Structures for Climate Insight](https://mlsanity.com/paper/2507.11085/) (2025)
- [Hide and Seek: Investigating Redundancy in Earth Observation Imagery](https://mlsanity.com/paper/2603.13524/) (2026)
- [Now We Know? A Systematic Comparison of TerraMind and THOR](https://mlsanity.com/paper/2607.18504/) (2026)
- [Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams](https://mlsanity.com/paper/2608.00012/) (2026)
- [Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference](https://mlsanity.com/paper/2608.15282/) (2026)

## Abstract (arXiv)

Forests are vital to ecosystems, supporting biodiversity and essential services, but are rapidly changing due to land use and climate change. Understanding and mitigating negative effects requires parsing data on forests at global scale from a broad array of sensory modalities, and using them in diverse forest monitoring applications. Such diversity in data and applications can be effectively addressed through the development of a large, pre-trained foundation model that serves as a versatile base for various downstream tasks. However, remote sensing modalities, which are an excellent fit for several forest management tasks, are particularly challenging considering the variation in environmental conditions, object scales, image acquisition modes, spatio-temporal resolutions, etc. With that in mind, we present the first unified Forest Monitoring Benchmark (FoMo-Bench), carefully constructed to evaluate foundation models with such flexibility. FoMo-Bench consists of 15 diverse datasets encompassing satellite, aerial, and inventory data, covering a variety of geographical regions, and including multispectral, red-green-blue, synthetic aperture radar and LiDAR data with various temporal, spatial and spectral resolutions. FoMo-Bench includes multiple types of forest-monitoring tasks, spanning classification, segmentation, and object detection. To enhance task and geographic diversity in FoMo-Bench, we introduce TalloS, a global dataset combining satellite imagery with ground-based annotations for tree species classification across 1,000+ categories and hierarchical taxonomic levels. Finally, we propose FoMo-Net, a pre-training framework to develop foundation models with the capacity to process any combination of commonly used modalities and spectral bands in remote sensing.