mlsanityData to 14 Oct 2026

Prospective Learning: Learning for a Dynamic Future

arXiv 2411.00109 ↗First posted 31 Oct 2024stat.ML cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In real-world applications, the distribution of the data, and our goals, evolve over time. The prevailing theoretical framework for studying machine learning, namely probably approximately correct (PAC) learning, largely ignores time. As a consequence, existing strategies to address the dynamic nature of data and goals exhibit poor real-world performance. This paper develops a theoretical framework called "Prospective Learning" that is tailored for situations when the optimal hypothesis changes over time. In PAC learning, empirical risk minimization (ERM) is known to be consistent. We develop a learner called Prospective ERM, which returns a sequence of predictors that make predictions on future data. We prove that the risk of prospective ERM converges to the Bayes risk under certain assumptions on the stochastic process generating the data. Prospective ERM, roughly speaking, incorporates time as an input in addition to the data. We show that standard ERM as done in PAC learning, without incorporating time, can result in failure to learn when distributions are dynamic. Numerical experiments illustrate that prospective ERM can learn synthetic and visual recognition problems constructed from MNIST and CIFAR-10. Code at https://github.com/neurodata/prolearn.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 31 of its 79 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

31 of 79 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2018Neural Ordinary Differential Equationscited by 3,779
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  8. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  10. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  11. 2019Three scenarios for continual learningcited by 614
  12. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
19 more
  1. 2019A Baseline for Few-Shot Image Classificationcited by 298
  2. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  3. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  4. 2020Continual Prototype Evolution: Learning Online from Non-Stationary Data Streamscited by 144
  5. 2019Online Meta-Learningcited by 137
  6. 2019Meta-Q-Learningcited by 62
  7. 2018Task Agnostic Continual Learning Using Online Variational Bayescited by 61
  8. 2021Out of Distribution Generalization in Machine Learningcited by 60
  9. 2021Smoothed Analysis with Adaptive Adversariescited by 57
  10. 2021Model Zoo: A Growing "Brain" That Learns Continuallycited by 54
  11. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  12. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 35
  13. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 24
  14. 2017Learning Whenever Learning is Possible: Universal Learning under General Stochastic Processescited by 23
  15. 2022Machine learning and invariant theorycited by 13
  16. 2024On the Performance of Empirical Risk Minimization with Smoothed Datacited by 13
  17. 2022The Value of Out-of-Distribution Datacited by 12
  18. 2022Prospective Learning: Principled Extrapolation to the Futurecited by 6
  19. 2022Time-Varying Propensity Score to Bridge the Gap between the Past and Presentcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Prospective Learning in Retrospectcited by 1
  2. 2025Learning Model Successorscited by 0
  3. 2025How Patterns Dictate Learnability in Sequential Datacited by 0
  4. 2025Optimal control of the future via prospective learning with controlcited by 0