# SHRUG-FM: Reliability-Aware Foundation Models for Earth Observation

arXiv: https://arxiv.org/abs/2511.10370 · HTML: https://mlsanity.com/paper/2511.10370/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 13 Nov 2025.

## Builds on (most cited first, 30 of 34)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023)
- [Foundation Models for Generalist Geospatial Artificial Intelligence](https://mlsanity.com/paper/2310.18660/) (2023)
- [Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications](https://mlsanity.com/paper/2412.02732/) (2024)
- [AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data](https://mlsanity.com/paper/2507.22291/) (2025)
- [Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers](https://mlsanity.com/paper/1805.08206/) (2018)
- [Machine Learning with a Reject Option: A survey](https://mlsanity.com/paper/2107.11277/) (2021)
- [TerraMind: Large-Scale Generative Multimodality for Earth Observation](https://mlsanity.com/paper/2504.11171/) (2025)
- [PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models](https://mlsanity.com/paper/2412.04204/) (2024)
- [TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis](https://mlsanity.com/paper/2506.20380/) (2025)
- [REOBench: Benchmarking Robustness of Earth Observation Foundation Models](https://mlsanity.com/paper/2505.16793/) (2025)
- [EarthPT: a time series foundation model for Earth Observation](https://mlsanity.com/paper/2309.07207/) (2023)
- [Distribution Shifts at Scale: Out-of-distribution Detection in Earth Observation](https://mlsanity.com/paper/2412.13394/) (2024)
- [Analyzing Local Representations of Self-supervised Vision Transformers](https://mlsanity.com/paper/2401.00463/) (2023)
- [Spatially-Aware Evaluation of Segmentation Uncertainty](https://mlsanity.com/paper/2506.16589/) (2025)
- [Decision boundaries and convex hulls in the feature space that deep learning functions learn from images](https://mlsanity.com/paper/2202.04052/) (2022)
- [How Does the Spatial Distribution of Pre-training Data Affect Geospatial Foundation Models?](https://mlsanity.com/paper/2501.12535/) (2025)
- [ExEBench: Benchmarking Foundation Models on Extreme Earth Events](https://mlsanity.com/paper/2505.08529/) (2025)
- [Deep Ensembles Secretly Perform Empirical Bayes](https://mlsanity.com/paper/2501.17917/) (2025)
- [NCDD: Nearest Centroid Distance Deficit for Out-Of-Distribution Detection in Gastrointestinal Vision](https://mlsanity.com/paper/2412.01590/) (2024)

## Built on by (most cited first, 3 of 3)

- [Spatial Representation Learning Beyond Pixels: Unifying Raster Data and Vector Semantics for Human-Centric Geospatial Foundation Models](https://mlsanity.com/paper/2606.02374/) (2026)
- [Beyond Accuracy: Assessing Calibration of Geospatial Foundation Models and Their Sensitivity to Distribution Shifts](https://mlsanity.com/paper/2608.16614/) (2026)
- [Transferability and operational reliability of a Prithvi crop classification foundation model under phenological and geographic shift across three continents](https://mlsanity.com/paper/2610.08810/) (2026)

## Abstract (arXiv)

Geospatial foundation models (GFMs) for Earth observation often fail to perform reliably in environments underrepresented during pretraining. We introduce SHRUG-FM, a framework for reliability-aware prediction that enables GFMs to identify and abstain from likely failures. Our approach integrates three complementary signals: geophysical out-of-distribution (OOD) detection in the input space, OOD detection in the embedding space, and task-specific predictive uncertainty. We evaluate SHRUG-FM across three high-stakes rapid-mapping tasks: burn scar segmentation, flood mapping, and landslide detection. Our results show that SHRUG-FM consistently reduces prediction risk on retained samples, outperforming established single-signal baselines like predictive entropy. Crucially, by utilizing a shallow "glass-box" decision tree for signal fusion, SHRUG-FM provides interpretable abstention thresholds. It builds a pathway toward safer and more interpretable deployment of GFMs in climate-sensitive applications, bridging the gap between benchmark performance and real-world reliability.