# The data synergy effects of time-series deep learning models in hydrology

arXiv: https://arxiv.org/abs/2101.01876 · HTML: https://mlsanity.com/paper/2101.01876/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 80 references that are in the map.
- First posted on arXiv 6 Jan 2021.

## Builds on (most cited first, 9 of 9)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets](https://mlsanity.com/paper/1907.08456/) (2019)
- [A trans-disciplinary review of deep learning research for water resources scientists](https://mlsanity.com/paper/1712.02162/) (2017)
- [A Comprehensive Review of Deep Learning Applications in Hydrology and Water Resources](https://mlsanity.com/paper/2007.12269/) (2020)
- [Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales](https://mlsanity.com/paper/1912.08949/) (2019)
- [From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling](https://mlsanity.com/paper/2007.15751/) (2020)
- [Prolongation of SMAP to Spatio-temporally Seamless Coverage of Continental US Using a Deep Learning Neural Network](https://mlsanity.com/paper/1707.06611/) (2017)
- [Rainfall-Runoff Prediction at Multiple Timescales with a Single Long Short-Term Memory Network](https://mlsanity.com/paper/2010.07921/) (2020)
- [The Proper Care and Feeding of CAMELS: How Limited Training Data Affects Streamflow Prediction](https://mlsanity.com/paper/1911.07249/) (2019)

## Built on by (most cited first, 10 of 10)

- [Differentiable, learnable, regionalized process-based models with physical outputs can approach state-of-the-art hydrologic prediction accuracy](https://mlsanity.com/paper/2203.14827/) (2022)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Probing the limit of hydrologic predictability with the Transformer network](https://mlsanity.com/paper/2306.12384/) (2023)
- [Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources](https://mlsanity.com/paper/2308.09766/) (2023)
- [Differentiable modeling to unify machine learning and physical models and advance Geosciences](https://mlsanity.com/paper/2301.04027/) (2023)
- [Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals](https://mlsanity.com/paper/2404.13885/) (2024)
- [Utilizing Earth Foundation Models to Enhance the Simulation Performance of Hydrological Models with AlphaEarth Embeddings](https://mlsanity.com/paper/2601.01558/) (2026)
- [Update hydrological states or meteorological forcings? Comparing data assimilation methods for differentiable hydrologic models](https://mlsanity.com/paper/2502.16444/) (2025)
- [Distinct hydrologic response patterns and trends worldwide revealed by physics-embedded learning](https://mlsanity.com/paper/2504.10707/) (2025)
- [Rethinking deep learning: linear regression remains a key benchmark in predicting terrestrial water storage](https://mlsanity.com/paper/2510.10799/) (2025)

## Abstract (arXiv)

When fitting statistical models to variables in geoscientific disciplines such as hydrology, it is a customary practice to regionalize - to divide a large spatial domain into multiple regions and study each region separately - instead of fitting a single model on the entire data (also known as unification). Traditional wisdom in these fields suggests that models built for each region separately will have higher performance because of homogeneity within each region. However, by partitioning the training data, each model has access to fewer data points and cannot learn from commonalities between regions. Here, through two hydrologic examples (soil moisture and streamflow), we argue that unification can often significantly outperform regionalization in the era of big data and deep learning (DL). Common DL architectures, even without bespoke customization, can automatically build models that benefit from regional commonality while accurately learning region-specific differences. We highlight an effect we call data synergy, where the results of the DL models improved when data were pooled together from characteristically different regions. In fact, the performance of the DL models benefited from more diverse rather than more homogeneous training data. We hypothesize that DL models automatically adjust their internal representations to identify commonalities while also providing sufficient discriminatory information to the model. The results here advocate for pooling together larger datasets, and suggest the academic community should place greater emphasis on data sharing and compilation.