# Online Deep Learning: Learning Deep Neural Networks on the Fly

arXiv: https://arxiv.org/abs/1711.03705 · HTML: https://mlsanity.com/paper/1711.03705/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2017.

## Built on by (most cited first, 30 of 86)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Continual Backprop: Stochastic Gradient Descent with Persistent Randomness](https://mlsanity.com/paper/2108.06325/) (2021)
- [Learning Fast and Slow for Online Time Series Forecasting](https://mlsanity.com/paper/2202.11672/) (2022)
- [DualNet: Continual Learning, Fast and Slow](https://mlsanity.com/paper/2110.00175/) (2021)
- [Debiasing Multimodal Large Language Models via Penalization of Language Priors](https://mlsanity.com/paper/2403.05262/) (2024)
- [Trompt: Towards a Better Deep Neural Network for Tabular Data](https://mlsanity.com/paper/2305.18446/) (2023)
- [FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations](https://mlsanity.com/paper/2203.13789/) (2022)
- [On Anytime Learning at Macroscale](https://mlsanity.com/paper/2106.09563/) (2021)
- [Unsupervised Continual Learning in Streaming Environments](https://mlsanity.com/paper/2109.09282/) (2021)
- [Class-Incremental Experience Replay for Continual Learning under Concept Drift](https://mlsanity.com/paper/2104.11861/) (2021)
- [A Survey on Semi-Supervised Learning for Delayed Partially Labelled Data Streams](https://mlsanity.com/paper/2106.09170/) (2021)
- [Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction](https://mlsanity.com/paper/2202.06472/) (2022)
- [AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges](https://mlsanity.com/paper/2304.04661/) (2023)
- [Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN](https://mlsanity.com/paper/2310.09163/) (2023)
- [Concept Drift Detection from Multi-Class Imbalanced Data Streams](https://mlsanity.com/paper/2104.10228/) (2021)
- [Adaptive Model Pooling for Online Deep Anomaly Detection from a Complex Evolving Data Stream](https://mlsanity.com/paper/2206.04792/) (2022)
- [ORFit: One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares](https://mlsanity.com/paper/2207.13853/) (2022)
- [Adaptive Online Incremental Learning for Evolving Data Streams](https://mlsanity.com/paper/2201.01633/) (2022)
- [KEEP: An Industrial Pre-Training Framework for Online Recommendation via Knowledge Extraction and Plugging](https://mlsanity.com/paper/2208.10174/) (2022)
- [GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data](https://mlsanity.com/paper/2309.17130/) (2023)
- [Untargeted White-box Adversarial Attack with Heuristic Defence Methods in Real-time Deep Learning based Network Intrusion Detection System](https://mlsanity.com/paper/2310.03334/) (2023)
- [Bridging Streaming Continual Learning via In-Context Large Tabular Models](https://mlsanity.com/paper/2512.11668/) (2025)
- [NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models](https://mlsanity.com/paper/2108.03434/) (2021)
- [Online Vertical Federated Learning for Cooperative Spectrum Sensing](https://mlsanity.com/paper/2312.11363/) (2023)
- [This Too Shall Pass: Removing Stale Observations in Dynamic Bayesian Optimization](https://mlsanity.com/paper/2405.14540/) (2024)
- [Act Now: A Novel Online Forecasting Framework for Large-Scale Streaming Data](https://mlsanity.com/paper/2412.00108/) (2024)
- [Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback](https://mlsanity.com/paper/2108.06167/) (2021)
- [Decoding ECoG signal into 3D hand translation using deep learning](https://mlsanity.com/paper/2110.03528/) (2021)
- [CAIPI in Practice: Towards Explainable Interactive Medical Image Classification](https://mlsanity.com/paper/2204.02661/) (2022)
- [Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts](https://mlsanity.com/paper/2303.05155/) (2023)

## Abstract (arXiv)

Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch learning setting, which requires the entire training data to be made available prior to the learning task. This is not scalable for many real-world scenarios where new data arrives sequentially in a stream form. We aim to address an open challenge of "Online Deep Learning" (ODL) for learning DNNs on the fly in an online setting. Unlike traditional online learning that often optimizes some convex objective function with respect to a shallow model (e.g., a linear/kernel-based hypothesis), ODL is significantly more challenging since the optimization of the DNN objective function is non-convex, and regular backpropagation does not work well in practice, especially for online learning settings. In this paper, we present a new online deep learning framework that attempts to tackle the challenges by learning DNN models of adaptive depth from a sequence of training data in an online learning setting. In particular, we propose a novel Hedge Backpropagation (HBP) method for online updating the parameters of DNN effectively, and validate the efficacy of our method on large-scale data sets, including both stationary and concept drifting scenarios.