# MODL: Multilearner Online Deep Learning

arXiv: https://arxiv.org/abs/2405.18281 · HTML: https://mlsanity.com/paper/2405.18281/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 73 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 23 of 23)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Repulsive Deep Ensembles are Bayesian](https://mlsanity.com/paper/2106.11642/) (2021)
- [Generative models improve fairness of medical classifiers under distribution shifts](https://mlsanity.com/paper/2304.09218/) (2023)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Using stacking to average Bayesian predictive distributions](https://mlsanity.com/paper/1704.02030/) (2017)
- [Self-Supervised Deep Visual Odometry with Online Adaptation](https://mlsanity.com/paper/2005.06136/) (2020)
- [On-the-Fly Test-time Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2203.05574/) (2022)
- [Physics-Guided Deep Neural Networks for Power Flow Analysis](https://mlsanity.com/paper/2002.00097/) (2020)
- [Online Cascade Learning for Efficient Inference over Streams](https://mlsanity.com/paper/2402.04513/) (2024)
- [ProtoRes: Proto-Residual Network for Pose Authoring via Learned Inverse Kinematics](https://mlsanity.com/paper/2106.01981/) (2021)
- [Brain Networks and Intelligence: A Graph Neural Network Based Approach to Resting State fMRI Data](https://mlsanity.com/paper/2311.03520/) (2023)
- [ORFit: One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares](https://mlsanity.com/paper/2207.13853/) (2022)
- [Diversity regularization in deep ensembles](https://mlsanity.com/paper/1802.07881/) (2018)
- [Efficient Methods for Online Multiclass Logistic Regression](https://mlsanity.com/paper/2110.03020/) (2021)
- [DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks](https://mlsanity.com/paper/2405.15805/) (2024)
- [Stochastic Online Optimization using Kalman Recursion](https://mlsanity.com/paper/2002.03636/) (2020)
- [Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts](https://mlsanity.com/paper/2303.05155/) (2023)
- [Burn After Reading: Online Adaptation for Cross-domain Streaming Data](https://mlsanity.com/paper/2112.04345/) (2021)
- [Simulation-based stacking](https://mlsanity.com/paper/2310.17009/) (2023)
- [Deep learning enhanced Rydberg multifrequency microwave recognition](https://mlsanity.com/paper/2202.13617/) (2022)
- [Personalized Negative Reservoir for Incremental Learning in Recommender Systems](https://mlsanity.com/paper/2403.03993/) (2024)

## Built on by (most cited first, 1 of 1)

- [Bayesian Ensembling: Insights from Online Optimization and Empirical Bayes](https://mlsanity.com/paper/2505.15638/) (2025)

## Abstract (arXiv)

Online deep learning tackles the challenge of learning from data streams by balancing two competing goals: fast learning and deep learning. However, existing research primarily emphasizes deep learning solutions, which are more adept at handling the ``deep'' aspect than the ``fast'' aspect of online learning. In this work, we introduce an alternative paradigm through a hybrid multilearner approach. We begin by developing a fast online logistic regression learner, which operates without relying on backpropagation. It leverages closed-form recursive updates of model parameters, efficiently addressing the fast learning component of the online learning challenge. This approach is further integrated with a cascaded multilearner design, where shallow and deep learners are co-trained in a cooperative, synergistic manner to solve the online learning problem. We demonstrate that this approach achieves state-of-the-art performance on standard online learning datasets. We make our code available: https://github.com/AntonValk/MODL