mlsanityData to 7 Oct 2026

Rethinking deep learning: linear regression remains a key benchmark in predicting terrestrial water storage

arXiv 2510.10799 ↗First posted 12 Oct 2025cs.LG physics.ao-ph physics.geo-phcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in machine learning such as Long Short-Term Memory (LSTM) models and Transformers have been widely adopted in hydrological applications, demonstrating impressive performance amongst deep learning models and outperforming physical models in various tasks. However, their superiority in predicting land surface states such as terrestrial water storage (TWS) that are dominated by many factors such as natural variability and human driven modifications remains unclear. Here, using the open-access, globally representative HydroGlobe dataset - comprising a baseline version derived solely from a land surface model simulation and an advanced version incorporating multi-source remote sensing data assimilation - we show that linear regression is a robust benchmark, outperforming the more complex LSTM and Temporal Fusion Transformer for TWS prediction. Our findings highlight the importance of including traditional statistical models as benchmarks when developing and evaluating deep learning models. Additionally, we emphasize the critical need to establish globally representative benchmark datasets that capture the combined impact of natural variability and human interventions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 70 references that are in the map. First posted on arXiv 12 Oct 2025.

Builds on

26 of 70 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  8. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  9. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  10. 2021Scaling Vision Transformerscited by 993
  11. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  12. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
14 more
  1. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
  2. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  3. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
  4. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 143
  5. 2024Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applicationscited by 139
  6. 2019Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasetscited by 68
  7. 2022Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practicescited by 54
  8. 2021Do We Really Need Deep Learning Models for Time Series Forecasting?cited by 45
  9. 2019Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scalescited by 27
  10. 2024PhilEO Bench: Evaluating Geo-Spatial Foundation Modelscited by 25
  11. 2021The data synergy effects of time-series deep learning models in hydrologycited by 10
  12. 2023Probing the limit of hydrologic predictability with the Transformer networkcited by 9
  13. 2024SatVision-TOA: A Geospatial Foundation Model for Coarse-Resolution All-Sky Remote Sensing Imagerycited by 7
  14. 2023Temporal Fusion Transformers for Streamflow Prediction: Value of Combining Attention with Recurrencecited by 4

Built on by

Not cited yet.