# Improving EO Foundation Models with Confidence Assessment for enhanced Semantic segmentation

arXiv: https://arxiv.org/abs/2406.18279 · HTML: https://mlsanity.com/paper/2406.18279/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 7 of 7)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Addressing Failure Prediction by Learning Model Confidence](https://mlsanity.com/paper/1910.04851/) (2019)
- [Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation](https://mlsanity.com/paper/2012.06575/) (2020)
- [Pixel-wise Anomaly Detection in Complex Driving Scenes](https://mlsanity.com/paper/2103.05445/) (2021)
- [PhilEO Bench: Evaluating Geo-Spatial Foundation Models](https://mlsanity.com/paper/2401.04464/) (2024)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [Uncertainty Measures and Prediction Quality Rating for the Semantic Segmentation of Nested Multi Resolution Street Scene Images](https://mlsanity.com/paper/1904.04516/) (2019)

## Abstract (arXiv)

Confidence assessments of semantic segmentation algorithms are important. Ideally, deep learning models should have the ability to predict in advance whether their output is likely to be incorrect. Assessing the confidence levels of model predictions in Earth Observation (EO) classification is essential, as it can enhance semantic segmentation performance and help prevent further exploitation of the results in case of erroneous prediction. The model we developed, Confidence Assessment for enhanced Semantic segmentation (CAS), evaluates confidence at both the segment and pixel levels, providing both labels and confidence scores as output. Our model, CAS, identifies segments with incorrect predicted labels using the proposed combined confidence metric, refines the model, and enhances its performance. This work has significant applications, particularly in evaluating EO Foundation Models on semantic segmentation downstream tasks, such as land cover classification using Sentinel-2 satellite data. The evaluation results show that this strategy is effective and that the proposed model CAS outperforms other baseline models.