# A Retrospective of the Tutorial on Opportunities and Challenges of Online Deep Learning

arXiv: https://arxiv.org/abs/2405.17222 · HTML: https://mlsanity.com/paper/2405.17222/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 47 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [River: machine learning for streaming data in Python](https://mlsanity.com/paper/2012.04740/) (2020)
- [A Confidence-Based Approach for Balancing Fairness and Accuracy](https://mlsanity.com/paper/1601.05764/) (2016)
- [Scikit-Multiflow: A Multi-output Streaming Framework](https://mlsanity.com/paper/1807.04662/) (2018)
- [FAHT: An Adaptive Fairness-aware Decision Tree Classifier](https://mlsanity.com/paper/1907.07237/) (2019)
- [Challenges in Benchmarking Stream Learning Algorithms with Real-world Data](https://mlsanity.com/paper/2005.00113/) (2020)
- [FNNC: Achieving Fairness through Neural Networks](https://mlsanity.com/paper/1811.00247/) (2018)
- [Fairness-enhancing interventions in stream classification](https://mlsanity.com/paper/1907.07223/) (2019)

## Abstract (arXiv)

Machine learning algorithms have become indispensable in today's world. They support and accelerate the way we make decisions based on the data at hand. This acceleration means that data structures that were valid at one moment could no longer be valid in the future. With these changing data structures, it is necessary to adapt machine learning (ML) systems incrementally to the new data. This is done with the use of online learning or continuous ML technologies. While deep learning technologies have shown exceptional performance on predefined datasets, they have not been widely applied to online, streaming, and continuous learning. In this retrospective of our tutorial titled Opportunities and Challenges of Online Deep Learning held at ECML PKDD 2023, we provide a brief overview of the opportunities but also the potential pitfalls for the application of neural networks in online learning environments using the frameworks River and Deep-River.