# Bayesian Incremental Learning for Deep Neural Networks

arXiv: https://arxiv.org/abs/1802.07329 · HTML: https://mlsanity.com/paper/1802.07329/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2018.

## Built on by (most cited first, 6 of 6)

- [Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting](https://mlsanity.com/paper/2102.01906/) (2021)
- [A Continual Learning Framework for Adaptive Defect Classification and Inspection](https://mlsanity.com/paper/2203.08796/) (2022)
- [Towards Zero-shot Language Modeling](https://mlsanity.com/paper/2108.03334/) (2021)
- [Federated Bayesian Deep Learning: The Application of Statistical Aggregation Methods to Bayesian Models](https://mlsanity.com/paper/2403.15263/) (2024)
- [Joint Text and Label Generation for Spoken Language Understanding](https://mlsanity.com/paper/2105.05052/) (2021)
- [FastODT: A tree-based framework for efficient continual learning](https://mlsanity.com/paper/2603.13276/) (2026)

## Abstract (arXiv)

In industrial machine learning pipelines, data often arrive in parts. Particularly in the case of deep neural networks, it may be too expensive to train the model from scratch each time, so one would rather use a previously learned model and the new data to improve performance. However, deep neural networks are prone to getting stuck in a suboptimal solution when trained on only new data as compared to the full dataset. Our work focuses on a continuous learning setup where the task is always the same and new parts of data arrive sequentially. We apply a Bayesian approach to update the posterior approximation with each new piece of data and find this method to outperform the traditional approach in our experiments.