# Evaluating and Characterizing Incremental Learning from Non-Stationary Data

arXiv: https://arxiv.org/abs/1806.06610 · HTML: https://mlsanity.com/paper/1806.06610/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2018.

## Built on by (most cited first, 3 of 3)

- [Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems](https://mlsanity.com/paper/2105.01064/) (2021)
- [Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems](https://mlsanity.com/paper/2209.01143/) (2022)
- [Fast Evaluation of DNN for Past Dataset in Incremental Learning](https://mlsanity.com/paper/2405.06296/) (2024)

## Abstract (arXiv)

Incremental learning from non-stationary data poses special challenges to the field of machine learning. Although new algorithms have been developed for this, assessment of results and comparison of behaviors are still open problems, mainly because evaluation metrics, adapted from more traditional tasks, can be ineffective in this context. Overall, there is a lack of common testing practices. This paper thus presents a testbed for incremental non-stationary learning algorithms, based on specially designed synthetic datasets. Also, test results are reported for some well-known algorithms to show that the proposed methodology is effective at characterizing their strengths and weaknesses. It is expected that this methodology will provide a common basis for evaluating future contributions in the field.