# Differentiable modeling to unify machine learning and physical models and advance Geosciences

arXiv: https://arxiv.org/abs/2301.04027 · HTML: https://mlsanity.com/paper/2301.04027/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 185 references that are in the map.
- First posted on arXiv 10 Jan 2023.

## Builds on (most cited first, 30 of 40)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets](https://mlsanity.com/paper/1907.08456/) (2019)
- [Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport](https://mlsanity.com/paper/1912.02968/) (2019)
- [On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence](https://mlsanity.com/paper/2207.04630/) (2022)
- [A trans-disciplinary review of deep learning research for water resources scientists](https://mlsanity.com/paper/1712.02162/) (2017)
- [Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability](https://mlsanity.com/paper/1912.01752/) (2019)
- [Deep Learning of Subsurface Flow via Theory-guided Neural Network](https://mlsanity.com/paper/1911.00103/) (2019)
- [Crop Yield Prediction Using Deep Neural Networks](https://mlsanity.com/paper/1902.02860/) (2019)
- [Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales](https://mlsanity.com/paper/1912.08949/) (2019)
- [From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling](https://mlsanity.com/paper/2007.15751/) (2020)
- [Hybrid FEM-NN models: Combining artificial neural networks with the finite element method](https://mlsanity.com/paper/2101.00962/) (2021)
- [Physics guided machine learning using simplified theories](https://mlsanity.com/paper/2012.13343/) (2020)
- [Differentiable, learnable, regionalized process-based models with physical outputs can approach state-of-the-art hydrologic prediction accuracy](https://mlsanity.com/paper/2203.14827/) (2022)
- [Coupling Machine Learning and Crop Modeling Improves Crop Yield Prediction in the US Corn Belt](https://mlsanity.com/paper/2008.04060/) (2020)
- [Physics Guided Machine Learning Methods for Hydrology](https://mlsanity.com/paper/2012.02854/) (2020)
- [Gradient Amplification: An efficient way to train deep neural networks](https://mlsanity.com/paper/2006.10560/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems](https://mlsanity.com/paper/2310.08644/) (2023)
- [Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources](https://mlsanity.com/paper/2308.09766/) (2023)
- [A Hybrid Modeling Framework for Crop Prediction Tasks via Dynamic Parameter Calibration and Multi-Task Learning](https://mlsanity.com/paper/2603.15411/) (2026)
- [Accurate, Efficient, and Explainable Deep Learning Approaches for Environmental Science Problems](https://mlsanity.com/paper/2605.19366/) (2026)

## Abstract (arXiv)

Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage large datasets. ML methods, especially deep networks, presented strong predictive skills yet lacked the ability to answer specific scientific questions. While various methods have been proposed for ML-physics integration, an important underlying theme -- differentiable modeling -- is not sufficiently recognized. Here we outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG). "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables, critically enabling the learning of high-dimensional unknown relationships. DG refers to a range of methods connecting varying amounts of prior knowledge to neural networks and training them together, capturing a different scope than physics-guided machine learning and emphasizing first principles. Preliminary evidence suggests DG offers better interpretability and causality than ML, improved generalizability and extrapolation capability, and strong potential for knowledge discovery, while approaching the performance of purely data-driven ML. DG models require less training data while scaling favorably in performance and efficiency with increasing amounts of data. With DG, geoscientists may be better able to frame and investigate questions, test hypotheses, and discover unrecognized linkages.