# From MNIST to ImageNet and Back: Benchmarking Continual Curriculum Learning

arXiv: https://arxiv.org/abs/2303.11076 · HTML: https://mlsanity.com/paper/2303.11076/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams](https://mlsanity.com/paper/2009.00919/) (2020)
- [Adversarial Continual Learning](https://mlsanity.com/paper/2003.09553/) (2020)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Don't forget, there is more than forgetting: new metrics for Continual Learning](https://mlsanity.com/paper/1810.13166/) (2018)
- [A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks](https://mlsanity.com/paper/2011.01844/) (2020)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data](https://mlsanity.com/paper/2108.09020/) (2021)
- [CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks](https://mlsanity.com/paper/2206.09059/) (2022)
- [Real-Time Evaluation in Online Continual Learning: A New Hope](https://mlsanity.com/paper/2302.01047/) (2023)
- [Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches](https://mlsanity.com/paper/1907.03799/) (2019)
- [Is Class-Incremental Enough for Continual Learning?](https://mlsanity.com/paper/2112.02925/) (2021)
- [Continual Barlow Twins: continual self-supervised learning for remote sensing semantic segmentation](https://mlsanity.com/paper/2205.11319/) (2022)
- [Continual Learning with Evolving Class Ontologies](https://mlsanity.com/paper/2210.04993/) (2022)
- [A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems](https://mlsanity.com/paper/2301.07799/) (2023)
- [Hierarchically Structured Task-Agnostic Continual Learning](https://mlsanity.com/paper/2211.07725/) (2022)

## Built on by (most cited first, 6 of 6)

- [Revisiting Catastrophic Forgetting in Large Language Model Tuning](https://mlsanity.com/paper/2406.04836/) (2024)
- [Data Distribution-based Curriculum Learning](https://mlsanity.com/paper/2402.07352/) (2024)
- [Meta-Learning Transformers to Improve In-Context Generalization](https://mlsanity.com/paper/2507.05019/) (2025)
- [Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Models](https://mlsanity.com/paper/2511.00335/) (2025)
- [TSN-Affinity: Similarity-Driven Parameter Reuse for Continual Offline Reinforcement Learning](https://mlsanity.com/paper/2604.25898/) (2026)
- [Towards Principled Continual Anomaly Detection: A Systematic Framework and Benchmark Scenarios](https://mlsanity.com/paper/2607.18289/) (2026)

## Abstract (arXiv)

Continual learning (CL) is one of the most promising trends in recent machine learning research. Its goal is to go beyond classical assumptions in machine learning and develop models and learning strategies that present high robustness in dynamic environments. The landscape of CL research is fragmented into several learning evaluation protocols, comprising different learning tasks, datasets, and evaluation metrics. Additionally, the benchmarks adopted so far are still distant from the complexity of real-world scenarios, and are usually tailored to highlight capabilities specific to certain strategies. In such a landscape, it is hard to objectively assess strategies. In this work, we fill this gap for CL on image data by introducing two novel CL benchmarks that involve multiple heterogeneous tasks from six image datasets, with varying levels of complexity and quality. Our aim is to fairly evaluate current state-of-the-art CL strategies on a common ground that is closer to complex real-world scenarios. We additionally structure our benchmarks so that tasks are presented in increasing and decreasing order of complexity -- according to a curriculum -- in order to evaluate if current CL models are able to exploit structure across tasks. We devote particular emphasis to providing the CL community with a rigorous and reproducible evaluation protocol for measuring the ability of a model to generalize and not to forget while learning. Furthermore, we provide an extensive experimental evaluation showing that popular CL strategies, when challenged with our benchmarks, yield sub-par performance, high levels of forgetting, and present a limited ability to effectively leverage curriculum task ordering. We believe that these results highlight the need for rigorous comparisons in future CL works as well as pave the way to design new CL strategies that are able to deal with more complex scenarios.