# Example-Based Concept Analysis Framework for Deep Weather Forecast Models

arXiv: https://arxiv.org/abs/2504.00831 · HTML: https://mlsanity.com/paper/2504.00831/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [Skillful Precipitation Nowcasting using Deep Generative Models of Radar](https://mlsanity.com/paper/2104.00954/) (2021)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [Dynamic Channel Pruning: Feature Boosting and Suppression](https://mlsanity.com/paper/1810.05331/) (2018)
- [Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making](https://mlsanity.com/paper/1902.02960/) (2019)
- [Intrinsic Probing through Dimension Selection](https://mlsanity.com/paper/2010.02812/) (2020)
- [Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability](https://mlsanity.com/paper/1912.01752/) (2019)
- [Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero](https://mlsanity.com/paper/2310.16410/) (2023)
- [A Tale of a Probe and a Parser](https://mlsanity.com/paper/2005.01641/) (2020)
- [Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks](https://mlsanity.com/paper/1705.01626/) (2017)
- [Gaussian Process Probes (GPP) for Uncertainty-Aware Probing](https://mlsanity.com/paper/2305.18213/) (2023)
- [Effective Training Strategies for Deep-learning-based Precipitation Nowcasting and Estimation](https://mlsanity.com/paper/2202.10555/) (2022)
- [Understanding Distributed Representations of Concepts in Deep Neural Networks without Supervision](https://mlsanity.com/paper/2312.17285/) (2023)
- [Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities Without Label Information](https://mlsanity.com/paper/2503.09068/) (2025)

## Abstract (arXiv)

To improve the trustworthiness of an AI model, finding consistent, understandable representations of its inference process is essential. This understanding is particularly important in high-stakes operations such as weather forecasting, where the identification of underlying meteorological mechanisms is as critical as the accuracy of the predictions. Despite the growing literature that addresses this issue through explainable AI, the applicability of their solutions is often limited due to their AI-centric development. To fill this gap, we follow a user-centric process to develop an example-based concept analysis framework, which identifies cases that follow a similar inference process as the target instance in a target model and presents them in a user-comprehensible format. Our framework provides the users with visually and conceptually analogous examples, including the probability of concept assignment to resolve ambiguities in weather mechanisms. To bridge the gap between vector representations identified from models and human-understandable explanations, we compile a human-annotated concept dataset and implement a user interface to assist domain experts involved in the the framework development.