# A Joint Learning Framework with Feature Reconstruction and Prediction for Incomplete Satellite Image Time Series in Agricultural Semantic Segmentation

arXiv: https://arxiv.org/abs/2505.19159 · HTML: https://mlsanity.com/paper/2505.19159/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 19 of 19)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [Self-Distillation Amplifies Regularization in Hilbert Space](https://mlsanity.com/paper/2002.05715/) (2020)
- [SAITS: Self-Attention-based Imputation for Time Series](https://mlsanity.com/paper/2202.08516/) (2022)
- [Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks](https://mlsanity.com/paper/2107.07933/) (2021)
- [ViTs for SITS: Vision Transformers for Satellite Image Time Series](https://mlsanity.com/paper/2301.04944/) (2023)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Data Augmentation techniques in time series domain: A survey and taxonomy](https://mlsanity.com/paper/2206.13508/) (2022)
- [Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder](https://mlsanity.com/paper/2102.01331/) (2021)
- [Convolutional Gated Recurrent Networks for Video Segmentation](https://mlsanity.com/paper/1611.05435/) (2016)
- [A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning](https://mlsanity.com/paper/2204.00951/) (2022)
- [Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation](https://mlsanity.com/paper/2008.04882/) (2020)
- [Classification of Large-Scale High-Resolution SAR Images with Deep Transfer Learning](https://mlsanity.com/paper/2001.01425/) (2020)
- [Data Augmentation for Time-Series Classification: An Extensive Empirical Study and Comprehensive Survey](https://mlsanity.com/paper/2310.10060/) (2023)
- [U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical Satellite Time Series](https://mlsanity.com/paper/2305.13277/) (2023)
- [Fusing Optical and SAR time series for LAI gap filling with multioutput Gaussian processes](https://mlsanity.com/paper/2012.02998/) (2020)
- [Revisiting the Encoding of Satellite Image Time Series](https://mlsanity.com/paper/2305.02086/) (2023)

## Built on by (most cited first, 1 of 1)

- [FarmMind: Reasoning-Query-Driven Dynamic Segmentation for Farmland Remote Sensing Images](https://mlsanity.com/paper/2601.22809/) (2026)

## Abstract (arXiv)

Satellite Image Time Series (SITS) is crucial for agricultural semantic segmentation. However, Cloud contamination introduces time gaps in SITS, disrupting temporal dependencies and causing feature shifts, leading to degraded performance of models trained on complete SITS. Existing methods typically address this by reconstructing the entire SITS before prediction or using data augmentation to simulate missing data. Yet, full reconstruction may introduce noise and redundancy, while the data-augmented model can only handle limited missing patterns, leading to poor generalization. We propose a joint learning framework with feature reconstruction and prediction to address incomplete SITS more effectively. During training, we simulate data-missing scenarios using temporal masks. The two tasks are guided by both ground-truth labels and the teacher model trained on complete SITS. The prediction task constrains the model from selectively reconstructing critical features from masked inputs that align with the teacher's temporal feature representations. It reduces unnecessary reconstruction and limits noise propagation. By integrating reconstructed features into the prediction task, the model avoids learning shortcuts and maintains its ability to handle varied missing patterns and complete SITS. Experiments on SITS from Hunan Province, Western France, and Catalonia show that our method improves mean F1-scores by 6.93% in cropland extraction and 7.09% in crop classification over baselines. It also generalizes well across satellite sensors, including Sentinel-2 and PlanetScope, under varying temporal missing rates and model backbones.