# Knowledge distillation as a pathway toward next-generation intelligent ecohydrological modeling systems

arXiv: https://arxiv.org/abs/2509.01972 · HTML: https://mlsanity.com/paper/2509.01972/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 50 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 13 of 13)

- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets](https://mlsanity.com/paper/1907.08456/) (2019)
- [Stochastic Training is Not Necessary for Generalization](https://mlsanity.com/paper/2109.14119/) (2021)
- [Deep Learning is Not So Mysterious or Different](https://mlsanity.com/paper/2503.02113/) (2025)
- [Understanding Generalization through Visualizations](https://mlsanity.com/paper/1906.03291/) (2019)
- [Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited](https://mlsanity.com/paper/2003.02139/) (2020)
- [Deep Learning of Subsurface Flow via Theory-guided Neural Network](https://mlsanity.com/paper/1911.00103/) (2019)
- [Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need](https://mlsanity.com/paper/2507.13966/) (2025)
- [Uncertainty Estimation with Deep Learning for Rainfall-Runoff Modelling](https://mlsanity.com/paper/2012.14295/) (2020)
- [A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems](https://mlsanity.com/paper/2310.08644/) (2023)
- [Towards Interpretable Physical-Conceptual Catchment-Scale Hydrological Modeling using the Mass-Conserving-Perceptron](https://mlsanity.com/paper/2401.14521/) (2024)
- [Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study](https://mlsanity.com/paper/2506.03931/) (2025)

## Abstract (arXiv)

Simulating ecohydrological processes is essential for understanding complex environmental systems and guiding sustainable management amid accelerating climate change and human pressures. Process-based models provide physical realism but can suffer from structural rigidity, high computational costs, and complex calibration, while machine learning (ML) methods are efficient and flexible yet often lack interpretability and transferability. We propose a unified three-phase framework that integrates process-based models with ML and progressively embeds them into artificial intelligence (AI) through knowledge distillation. Phase I, behavioral distillation, enhances process models via surrogate learning and model simplification to capture key dynamics at lower computational cost. Phase II, structural distillation, reformulates process equations as modular components within a graph neural network (GNN), enabling multiscale representation and seamless integration with ML models. Phase III, cognitive distillation, embeds expert reasoning and adaptive decision-making into intelligent modeling agents using the Eyes-Brain-Hands-Mouth architecture. Demonstrations for the Samish watershed highlight the framework's applicability to ecohydrological modeling, showing that it can reproduce process-based model outputs, improve predictive accuracy, and support scenario-based decision-making. The framework offers a scalable and transferable pathway toward next-generation intelligent ecohydrological modeling systems, with the potential extension to other process-based domains.