mlsanityData to 7 Oct 2026

Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series

arXiv 2603.02620 ↗First posted 3 Mar 2026cs.LG q-fin.CPcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale volatility forecasting for S&P 500 stocks, we show that different model-training-pipeline pairs with identical test loss learn qualitatively different functions. Across architectures, predictive accuracy remains unchanged, yet optimizer choice reshapes non-linear response profiles and temporal dependence differently. These divergences have material consequences for decisions: volatility-ranked portfolios trace a near-vertical Sharpe-turnover frontier, with nearly 3× turnover dispersion at comparable Sharpe ratios. We conclude that in underspecified settings, optimization acts as a consequential source of inductive bias, thus model evaluation should extend beyond scalar loss to encompass functional and decision-level implications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 65 of its 114 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

65 of 114 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  5. 2019Invariant Risk Minimizationcited by 1,832
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  8. 2018On the Spectral Bias of Neural Networkscited by 1,321
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  10. 2019On the Convergence of Adam and Beyondcited by 834
  11. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  12. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
38 more
  1. 2018On Lazy Training in Differentiable Programmingcited by 671
  2. 2023A decoder-only foundation model for time-series forecastingcited by 650
  3. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  4. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  5. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  6. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  7. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  8. 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
  9. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 413
  10. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  11. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  12. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  13. 2017Implicit Regularization in Matrix Factorizationcited by 318
  14. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  15. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  16. 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 259
  17. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  18. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 248
  19. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  20. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  21. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 183
  22. 2019Predictive Multiplicity in Classificationcited by 162
  23. 2022Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stabilitycited by 151
  24. 2020The Break-Even Point on Optimization Trajectories of Deep Neural Networkscited by 149
  25. 2024Why Transformers Need Adam: A Hessian Perspectivecited by 123
  26. 2022Adaptive Gradient Methods at the Edge of Stabilitycited by 88
  27. 2019On the Existence of Simpler Machine Learning Modelscited by 87
  28. 2018On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Lengthcited by 85
  29. 2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 80
  30. 2024Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Modelscited by 78
  31. 2024Understanding Optimization in Deep Learning with Central Flowscited by 57
  32. 2024Amazing Things Come From Having Many Good Modelscited by 52
  33. 2021Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularizationcited by 48
  34. 2019Deep Learning in Asset Pricingcited by 41
  35. 2024Deconstructing What Makes a Good Optimizer for Language Modelscited by 40
  36. 2022Beyond the Edge of Stability via Two-step Gradient Updatescited by 38
  37. 2021Fast Sparse Decision Tree Optimization via Reference Ensemblescited by 37
  38. 2023A Path to Simpler Models Starts With Noisecited by 33

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.