mlsanityData to 14 Oct 2026

Staying Alive: Online Neural Network Maintenance and Systemic Drift

arXiv 2503.17681 ↗First posted 22 Mar 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present the Subset Extended Kalman Filter (SEKF) as a method to update previously trained model weights online rather than retraining or finetuning them when the system a model represents drifts away from the conditions under which it was trained. We identify the parameters to be updated using the gradient of the loss function and use the SEKF to update only these parameters. We compare finetuning and SEKF for online model maintenance in the presence of systemic drift through four dynamic regression case studies and find that the SEKF is able to maintain model accuracy as-well if not better than finetuning while requiring significantly less time per iteration, and less hyperparameter tuning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 72 references that are in the map. First posted on arXiv 22 Mar 2025.

Builds on

26 of 72 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  5. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  6. 2017Variational Continual Learningcited by 339
  7. 2014The Loss Surfaces of Multilayer Networkscited by 280
  8. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
  9. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  10. 2020Efficient Continual Learning with Modular Networks and Task-Driven Priorscited by 88
  11. 2022From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectorscited by 83
  12. 2024Continual Learning and Catastrophic Forgettingcited by 77
14 more
  1. 2023Continual Learning: Applications and the Road Forwardcited by 59
  2. 2021A study on the plasticity of neural networkscited by 44
  3. 2018SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradientcited by 34
  4. 2020Physics-constrained Deep Learning of Multi-zone Building Thermal Dynamicscited by 32
  5. 2023Low-rank extended Kalman filtering for online learning of neural networks from streaming datacited by 23
  6. 2021DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimizationcited by 19
  7. 2019Rapid online learning and robust recall in a neuromorphic olfactory circuitcited by 18
  8. 2024Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Resetcited by 17
  9. 2021Automatically detecting data drift in machine learning classifierscited by 15
  10. 2022ORFit: One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squarescited by 12
  11. 2021Kalman Bayesian Neural Networks for Closed-form Online Learningcited by 9
  12. 2021Online Learning for Recommendations at Grubhubcited by 7
  13. 2022Neural Network-based Flight Control Systems: Present and Futurecited by 6
  14. 2018Efficient Full-Matrix Adaptive Regularizationcited by 5

Built on by

Not cited yet.