# Distillation and Interpretability of Ensemble Forecasts of ENSO Phase using Entropic Learning

arXiv: https://arxiv.org/abs/2602.16857 · HTML: https://mlsanity.com/paper/2602.16857/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 53 references that are in the map.
- First posted on arXiv 15 Feb 2026.

## Builds on (most cited first, 8 of 8)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [ResoNet: Robust and Explainable ENSO Forecasts with Hybrid Convolution and Transformer Networks](https://mlsanity.com/paper/2312.10429/) (2023)
- [Towards Long-Range ENSO Prediction with an Explainable Deep Learning Model](https://mlsanity.com/paper/2503.19502/) (2025)
- [An entropy-optimal path to humble AI](https://mlsanity.com/paper/2506.17940/) (2025)

## Built on by (most cited first, 1 of 1)

- [On a joint simultaneous learning of relevant feature subsets and subspaces in regression-like problems](https://mlsanity.com/paper/2607.28080/) (2026)

## Abstract (arXiv)

This paper introduces a distillation framework for an ensemble of entropy-optimal Sparse Probabilistic Approximation (eSPA) models, trained exclusively on satellite-era observational and reanalysis data to predict ENSO phase up to 24 months in advance. While eSPA ensembles yield state-of-the-art forecast skill, they are harder to interpret than individual eSPA models. We show how to compress the ensemble into a compact set of "distilled" models by aggregating the structure of only those ensemble members that make correct predictions. This process yields a single, diagnostically tractable model for each forecast lead time that preserves forecast performance while also enabling diagnostics that are impractical to implement on the full ensemble. An analysis of the regime persistence of the distilled model "superclusters", as well as cross-lead clustering consistency, shows that the discretised system accurately captures the spatiotemporal dynamics of ENSO. By considering the effective dimension of the feature importance vectors, the complexity of the input space required for correct ENSO phase prediction is shown to peak when forecasts must cross the boreal spring predictability barrier. Spatial importance maps derived from the feature importance vectors are introduced to identify where predictive information resides in each field and are shown to include known physical precursors at certain lead times. Case studies of key events are also presented, showing how fields reconstructed from distilled model centroids trace the evolution from extratropical and inter-basin precursors to the mature ENSO state. Overall, the distillation framework enables a rigorous investigation of long-range ENSO predictability that complements real-time data-driven operational forecasts.