# Curricular and Cyclical Loss for Time Series Learning Strategy

arXiv: https://arxiv.org/abs/2312.15853 · HTML: https://mlsanity.com/paper/2312.15853/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 26 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [An Improved Analysis of Stochastic Gradient Descent with Momentum](https://mlsanity.com/paper/2007.07989/) (2020)
- [When Do Curricula Work?](https://mlsanity.com/paper/2012.03107/) (2020)
- [Approaches and Applications of Early Classification of Time Series: A Review](https://mlsanity.com/paper/2005.02595/) (2020)
- [A Review of Deep Learning Methods for Irregularly Sampled Medical Time Series Data](https://mlsanity.com/paper/2010.12493/) (2020)
- [Self-supervised Learning of Geometrically Stable Features Through Probabilistic Introspection](https://mlsanity.com/paper/1804.01552/) (2018)
- [Pay Attention to Evolution: Time Series Forecasting with Deep Graph-Evolution Learning](https://mlsanity.com/paper/2008.12833/) (2020)
- [Deep Time Series Forecasting with Shape and Temporal Criteria](https://mlsanity.com/paper/2104.04610/) (2021)
- [TE-ESN: Time Encoding Echo State Network for Prediction Based on Irregularly Sampled Time Series Data](https://mlsanity.com/paper/2105.00412/) (2021)
- [On the Statistical Benefits of Curriculum Learning](https://mlsanity.com/paper/2111.07126/) (2021)
- [Sample Amplification: Increasing Dataset Size even when Learning is Impossible](https://mlsanity.com/paper/1904.12053/) (2019)
- [Cyclical Curriculum Learning](https://mlsanity.com/paper/2202.05531/) (2022)
- [Continuous Diagnosis and Prognosis by Controlling the Update Process of Deep Neural Networks](https://mlsanity.com/paper/2210.02719/) (2022)
- [Confidence-Guided Learning Process for Continuous Classification of Time Series](https://mlsanity.com/paper/2208.06883/) (2022)

## Built on by (most cited first, 1 of 1)

- [Review of Data-centric Time Series Analysis from Sample, Feature, and Period](https://mlsanity.com/paper/2404.16886/) (2024)

## Abstract (arXiv)

Time series widely exists in real-world applications and many deep learning models have performed well on it. Current research has shown the importance of learning strategy for models, suggesting that the benefit is the order and size of learning samples. However, no effective strategy has been proposed for time series due to its abstract and dynamic construction. Meanwhile, the existing one-shot tasks and continuous tasks for time series necessitate distinct learning processes and mechanisms. No all-purpose approach has been suggested. In this work, we propose a novel Curricular and CyclicaL loss (CRUCIAL) to learn time series for the first time. It is model- and task-agnostic and can be plugged on top of the original loss with no extra procedure. CRUCIAL has two characteristics: It can arrange an easy-to-hard learning order by dynamically determining the sample contribution and modulating the loss amplitude; It can manage a cyclically changed dataset and achieve an adaptive cycle by correlating the loss distribution and the selection probability. We prove that compared with monotonous size, cyclical size can reduce expected error. Experiments on 3 kinds of tasks and 5 real-world datasets show the benefits of CRUCIAL for most deep learning models when learning time series.