# Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data

arXiv: https://arxiv.org/abs/2305.12011 · HTML: https://mlsanity.com/paper/2305.12011/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 79 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images](https://mlsanity.com/paper/1805.06561/) (2018)
- [Gated Multimodal Units for Information Fusion](https://mlsanity.com/paper/1702.01992/) (2017)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [A Survey on Multi-view Learning](https://mlsanity.com/paper/1304.5634/) (2013)
- [End-to-End Attention-based Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1508.04395/) (2015)
- [Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series](https://mlsanity.com/paper/1811.10166/) (2018)
- [Self-attention for raw optical Satellite Time Series Classification](https://mlsanity.com/paper/1910.10536/) (2019)
- [Satellite Image Time Series Classification with Pixel-Set Encoders and Temporal Self-Attention](https://mlsanity.com/paper/1911.07757/) (2019)
- [Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders](https://mlsanity.com/paper/1802.02080/) (2018)
- [Gated Feedback Recurrent Neural Networks](https://mlsanity.com/paper/1502.02367/) (2015)
- [A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks](https://mlsanity.com/paper/1811.06031/) (2018)
- [Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning](https://mlsanity.com/paper/1802.00924/) (2018)
- [Crop mapping from image time series: deep learning with multi-scale label hierarchies](https://mlsanity.com/paper/2102.08820/) (2021)
- [Zero-Shot Deep Domain Adaptation](https://mlsanity.com/paper/1707.01922/) (2017)
- [Meta-Learning for Few-Shot Land Cover Classification](https://mlsanity.com/paper/2004.13390/) (2020)
- [TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation](https://mlsanity.com/paper/2111.02682/) (2021)
- [Time-Space tradeoff in deep learning models for crop classification on satellite multi-spectral image time series](https://mlsanity.com/paper/1901.10503/) (2019)
- [End-to-End Learned Early Classification of Time Series for In-Season Crop Type Mapping](https://mlsanity.com/paper/1901.10681/) (2019)
- [Generalized Classification of Satellite Image Time Series with Thermal Positional Encoding](https://mlsanity.com/paper/2203.09175/) (2022)
- [EuroCrops: A Pan-European Dataset for Time Series Crop Type Classification](https://mlsanity.com/paper/2106.08151/) (2021)
- [From parcel to continental scale -- A first European crop type map based on Sentinel-1 and LUCAS Copernicus in-situ observations](https://mlsanity.com/paper/2105.09261/) (2021)
- [Early Classification for Agricultural Monitoring from Satellite Time Series](https://mlsanity.com/paper/1908.10283/) (2019)
- [Early- and in-season crop type mapping without current-year ground truth: generating labels from historical information via a topology-based approach](https://mlsanity.com/paper/2110.10275/) (2021)
- [Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series](https://mlsanity.com/paper/2110.08187/) (2021)
- [Multimodal Crop Type Classification Fusing Multi-Spectral Satellite Time Series with Farmers Crop Rotations and Local Crop Distribution](https://mlsanity.com/paper/2208.10838/) (2022)

## Built on by (most cited first, 8 of 8)

- [Deep Learning for Satellite Image Time Series Analysis: A Review](https://mlsanity.com/paper/2404.03936/) (2024)
- [AgriFM: A Multi-source Temporal Remote Sensing Foundation Model for Agriculture Mapping](https://mlsanity.com/paper/2505.21357/) (2025)
- [FLAIR-HUB: Large-scale Multimodal Dataset for Land Cover and Crop Mapping](https://mlsanity.com/paper/2506.07080/) (2025)
- [SwissCrop25: A National Multi-Year Benchmark for Operational Crop Mapping](https://mlsanity.com/paper/2608.09497/) (2026)
- [Plantation Monitoring Using Drone Images: A Dataset and Performance Review](https://mlsanity.com/paper/2502.08233/) (2025)
- [Remote Sensing Based Crop Health Classification Using NDVI and Fully Connected Neural Networks](https://mlsanity.com/paper/2504.10522/) (2025)
- [T3S: Think in Thermal Time for Generalizable Crop Mapping from Satellite Image Time Series](https://mlsanity.com/paper/2506.12885/) (2025)
- [Attention-Based Ensemble Learning for Crop Classification Using Landsat 8-9 Fusion](https://mlsanity.com/paper/2506.18321/) (2025)

## Abstract (arXiv)

Accurate in-season crop type classification is crucial for the crop production estimation and monitoring of agricultural parcels. However, the complexity of the plant growth patterns and their spatio-temporal variability present significant challenges. While current deep learning-based methods show promise in crop type classification from single- and multi-modal time series, most existing methods rely on a single modality, such as satellite optical remote sensing data or crop rotation patterns. We propose a novel approach to fuse multimodal information into a model for improved accuracy and robustness across multiple years and countries. The approach relies on three modalities used: remote sensing time series from Sentinel-2 and Landsat 8 observations, parcel crop rotation and local crop distribution. To evaluate our approach, we release a new annotated dataset of 7.4 million agricultural parcels in France and Netherlands. We associate each parcel with time-series of surface reflectance (Red and NIR) and biophysical variables (LAI, FAPAR). Additionally, we propose a new approach to automatically aggregate crop types into a hierarchical class structure for meaningful model evaluation and a novel data-augmentation technique for early-season classification. Performance of the multimodal approach was assessed at different aggregation level in the semantic domain spanning from 151 to 8 crop types or groups. It resulted in accuracy ranging from 91% to 95% for NL dataset and from 85% to 89% for FR dataset. Pre-training on a dataset improves domain adaptation between countries, allowing for cross-domain zero-shot learning, and robustness of the performances in a few-shot setting from France to Netherlands. Our proposed approach outperforms comparable methods by enabling learning methods to use the often overlooked spatio-temporal context of parcels, resulting in increased preci...