mlsanityData to 14 Oct 2026

Hedging Memory Horizons for Non-Stationary Prediction via Online Aggregation

arXiv 2605.06541 ↗First posted 7 May 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study online prediction under distribution shift, where inputs arrive chronologically and outcomes are revealed only after prediction. In this setting, predictors must remain stable in quiet regimes yet adapt when regimes shift, and the right adaptation memory is unknown in advance. We propose MELO (Memory-hedged Exponentially Weighted Least-Squares Online aggregation), a model-agnostic method that hedges across adaptation scales: it wraps any non-anticipating base-predictor pool with exponentially weighted least-squares (EWLS) adaptation experts at multiple forgetting factors, and aggregates raw and EWLS-adapted forecasts with MLpol, a parameter-free online aggregation rule. Under boundedness conditions, we establish deterministic oracle inequalities showing that it competes with both the best raw predictor and the best bounded, time-varying affine combinations of the base predictions, up to a path-length-dependent tracking cost and a sublinear aggregation overhead. We evaluate MELO on French national electricity-load forecasting through the COVID-19 lockdown using no regime indicators, lockdown dates, or policy covariates. MELO reduces overall RMSE by 34.7% relative to base-only MLpol and achieves lower overall RMSE than a TabICL reference supplied with an external COVID policy-response covariate. Moreover, MELO requires only lightweight per-step recursive updates without model retraining.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 46 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

19 of 46 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  4. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  5. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  6. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  7. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 186
  8. 2018Adaptive Online Learning in Dynamic Environmentscited by 133
  9. 2014A Second-order Bound with Excess Lossescited by 95
  10. 2014Optimal learning with Bernstein Online Aggregationcited by 35
  11. 2025Battling the Non-stationarity in Time Series Forecasting via Test-time Adaptationcited by 35
  12. 2024TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarkscited by 33
7 more
  1. 2020Adaptive Methods for Short-Term Electricity Load Forecasting During COVID-19 Lockdown in Francecited by 23
  2. 2020Non-exponentially weighted aggregation: regret bounds for unbounded loss functionscited by 18
  3. 2025Lightweight Online Adaption for Time Series Foundation Model Forecastscited by 16
  4. 2025Improving Time Series Forecasting via Instance-aware Post-hoc Revisioncited by 15
  5. 2013Aggregation of Affine Estimatorscited by 4
  6. 2021Viking: Variational Bayesian Variance Trackingcited by 4
  7. 2023Online Ensemble Learning for Sector Rotation: A Gradient-Free Frameworkcited by 2

Built on by

Not cited yet.