# Adaptive prediction theory combining offline and online learning

arXiv: https://arxiv.org/abs/2512.00342 · HTML: https://mlsanity.com/paper/2512.00342/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Builds on (most cited first, 16 of 16)

- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Adaptive Online Learning in Dynamic Environments](https://mlsanity.com/paper/1810.10815/) (2018)
- [Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds](https://mlsanity.com/paper/2205.06908/) (2022)
- [Learning with Square Loss: Localization through Offset Rademacher Complexity](https://mlsanity.com/paper/1502.06134/) (2015)
- [Learning with little mixing](https://mlsanity.com/paper/2206.08269/) (2022)
- [Single Trajectory Nonparametric Learning of Nonlinear Dynamics](https://mlsanity.com/paper/2202.08311/) (2022)
- [On the Generalization for Transfer Learning: An Information-Theoretic Analysis](https://mlsanity.com/paper/2207.05377/) (2022)
- [Learning Dynamical Systems by Leveraging Data from Similar Systems](https://mlsanity.com/paper/2302.04344/) (2023)
- [Active Learning for Control-Oriented Identification of Nonlinear Systems](https://mlsanity.com/paper/2404.09030/) (2024)
- [On the speed of uniform convergence in Mercer's theorem](https://mlsanity.com/paper/2205.00487/) (2022)
- [Deep Model Predictive Control with Stability Guarantees](https://mlsanity.com/paper/2104.07171/) (2021)
- [Identification and Adaptation with Binary-Valued Observations under Non-Persistent Excitation Condition](https://mlsanity.com/paper/2107.03588/) (2021)
- [Rate-Optimal Non-Asymptotics for the Quadratic Prediction Error Method](https://mlsanity.com/paper/2404.07937/) (2024)
- [Online Residual Learning from Offline Experts for Pedestrian Tracking](https://mlsanity.com/paper/2409.04069/) (2024)
- [L1-Based Adaptive Identification with Saturated Observations](https://mlsanity.com/paper/2412.10695/) (2024)
- [Learning Linearized Models from Nonlinear Systems under Initialization Constraints with Finite Data](https://mlsanity.com/paper/2505.04954/) (2025)

## Abstract (arXiv)

Real-world intelligence systems usually operate by combining offline learning and online adaptation with highly correlated and non-stationary system data or signals, which, however, has rarely been investigated theoretically in the literature. This paper initiates a theoretical investigation on the prediction performance of a two-stage learning framework combining offline and online algorithms for a class of nonlinear stochastic dynamical systems. For the offline-learning phase, we establish an upper bound on the generalization error for approximate nonlinear-least-squares estimation under general datasets with strong correlation and distribution shift, leveraging the Kullback-Leibler divergence to quantify the distributional discrepancies. For the online-adaptation phase, we address, on the basis of the offline-trained model, the possible uncertain parameter drift in real-world target systems by proposing a meta-LMS prediction algorithm. This two-stage framework, integrating offline learning with online adaptation, demonstrates superior prediction performances compared with either purely offline or online methods. Both theoretical guarantees and empirical studies are provided.