mlsanityData to 7 Oct 2026

Discounted Adaptive Online Learning: Towards Better Regularization

arXiv 2402.02720 ↗First posted 5 Feb 2024cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

We study online learning in adversarial nonstationary environments. Since the future can be very different from the past, a critical challenge is to gracefully forget the history while new data comes in. To formalize this intuition, we revisit the discounted regret in online convex optimization, and propose an adaptive (i.e., instance optimal), FTRL-based algorithm that improves the widespread non-adaptive baseline -- gradient descent with a constant learning rate. From a practical perspective, this refines the classical idea of regularization in lifelong learning: we show that designing good regularizers can be guided by the principled theory of adaptive online optimization. Complementing this result, we also consider the (Gibbs and Cand`es, 2021)-style online conformal prediction problem, where the goal is to sequentially predict the uncertainty sets of a black-box machine learning model. We show that the FTRL nature of our algorithm can simplify the conventional gradient-descent-based analysis, leading to instance-dependent performance guarantees.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 39 of its 62 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

39 of 62 references.

References · 39

  1. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  2. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  3. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  4. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
  5. 2022Conformal prediction beyond exchangeabilitycited by 348
  6. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  7. 2023Understanding plasticity in neural networkscited by 170
  8. 2022Adaptive Conformal Predictions for Time Seriescited by 155
  9. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  10. 2018Adaptive Online Learning in Dynamic Environmentscited by 130
  11. 2018Black-Box Reductions for Parameter-free Online Learning in Banach Spacescited by 124
  12. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 123
27 more
  1. 2015Strongly Adaptive Online Learningcited by 102
  2. 2023A Definition of Continual Reinforcement Learningcited by 98
  3. 2020Dynamic Regret of Convex and Smooth Functionscited by 90
  4. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  5. 2016Scale-Free Online Learningcited by 83
  6. 2023Improved Online Conformal Prediction via Strongly Adaptive Online Learningcited by 76
  7. 2018Solving Imperfect-Information Games via Discounted Regret Minimizationcited by 68
  8. 2022Practical Adversarial Multivalid Conformal Predictioncited by 65
  9. 2020Lipschitz and Comparator-Norm Adaptivity in Online Learningcited by 57
  10. 2021Impossible Tuning Made Possible: A New Expert Algorithm and Its Applicationscited by 52
  11. 2023RDumb: A simple approach that questions our progress in continual test-time adaptationcited by 50
  12. 2016Improved Strongly Adaptive Online Learning using Coin Bettingcited by 48
  13. 2014Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximationscited by 46
  14. 2022Parameter-free Mirror Descentcited by 45
  15. 2023Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversioncited by 42
  16. 2021Optimal Dynamic Regret in Exp-Concave Online Learningcited by 39
  17. 2021Parameter-free Stochastic Optimization of Variationally Coherent Functionscited by 27
  18. 2022PDE-Based Optimal Strategy for Unconstrained Online Learningcited by 27
  19. 2022Optimal Dynamic Regret in Proper Online Learning with Strongly Convex Losses and Beyondcited by 27
  20. 2024Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguisecited by 23
  21. 2020Online mirror descent and dual averaging: keeping pace in the dynamic casecited by 21
  22. 2023Normalized Gradients for Allcited by 18
  23. 2019Prediction with Expert Advice: a PDE Perspectivecited by 16
  24. 2020Optimal anytime regret with two expertscited by 16
  25. 2023Unconstrained Dynamic Regret via Sparse Codingcited by 11
  26. 2023Improving Adaptive Online Learning Using Refined Discretizationcited by 8
  27. 2022On the Computational Efficiency of Adaptive and Dynamic Regret Minimizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 3.

12 papers.

  1. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learningcited by 16
  2. 2025On the Dynamic Regret of Following the Regularized Leader: Optimism with History Pruningcited by 5
  3. 2024Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimizationcited by 4
  4. 2024The Benefit of Being Bayesian in Online Conformal Predictioncited by 3
  5. 2025Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learningcited by 2
  6. 2025Model-free Online Learning for the Kalman Filter: Forgetting Factor and Logarithmic Regretcited by 2
  7. 2025Discounted Online Convex Optimization: Uniform Regret Across a Continuous Intervalcited by 2
  8. 2026Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizercited by 2
  9. 2025CONTINA: Confidence Interval for Traffic Demand Prediction with Coverage Guaranteecited by 0
  10. 2025Feature Fitted Online Conformal Prediction for Deep Time Series Forecasting Modelcited by 0
  11. 2026Elements of Conformal Predictioncited by 0
  12. 2026Parameter-Free and Group Conditional Online Conformal Predictioncited by 0