# Deep Time Series Models for Scarce Data

arXiv: https://arxiv.org/abs/2103.09348 · HTML: https://mlsanity.com/paper/2103.09348/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 43 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Builds on (most cited first, 3 of 3)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Multivariate Functional Principal Component Analysis for Data Observed on Different (Dimensional) Domains](https://mlsanity.com/paper/1509.02029/) (2015)
- [Remaining Useful Life Estimation Using Functional Data Analysis](https://mlsanity.com/paper/1904.06442/) (2019)

## Built on by (most cited first, 8 of 8)

- [Data Augmentation techniques in time series domain: A survey and taxonomy](https://mlsanity.com/paper/2206.13508/) (2022)
- [Weak-signal extraction enabled by deep-neural-network denoising of diffraction data](https://mlsanity.com/paper/2209.09247/) (2022)
- [WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generation](https://mlsanity.com/paper/2510.11839/) (2025)
- [M-Loss: Quantifying Model Merging Compatibility with Limited Unlabeled Data](https://mlsanity.com/paper/2602.08564/) (2026)
- [Rethinking Remaining Useful Life Prediction with Scarce Time Series Data: Regression under Indirect Supervision](https://mlsanity.com/paper/2504.09206/) (2025)
- [A complex network approach to time series analysis with application in diagnosis of neuromuscular disorders](https://mlsanity.com/paper/2108.06920/) (2021)
- [DAD4TS: Data-Augmentation-Oriented Diffusion Model for Time-Series Forecasting with Small-Scale Data](https://mlsanity.com/paper/2605.17866/) (2026)
- [Physically-Constrained Mamba-SDE for Remaining Useful Life Prediction under Irregular Observations](https://mlsanity.com/paper/2606.01894/) (2026)

## Abstract (arXiv)

Time series data have grown at an explosive rate in numerous domains and have stimulated a surge of time series modeling research. A comprehensive comparison of different time series models, for a considered data analytics task, provides useful guidance on model selection for data analytics practitioners. Data scarcity is a universal issue that occurs in a vast range of data analytics problems, due to the high costs associated with collecting, generating, and labeling data as well as some data quality issues such as missing data. In this paper, we focus on the temporal classification/regression problem that attempts to build a mathematical mapping from multivariate time series inputs to a discrete class label or a real-valued response variable. For this specific problem, we identify two types of scarce data: scarce data with small samples and scarce data with sparsely and irregularly observed time series covariates. Observing that all existing works are incapable of utilizing the sparse time series inputs for proper modeling building, we propose a model called sparse functional multilayer perceptron (SFMLP) for handling the sparsity in the time series covariates. The effectiveness of the proposed SFMLP under each of the two types of data scarcity, in comparison with the conventional deep sequential learning models (e.g., Recurrent Neural Network, and Long Short-Term Memory), is investigated through mathematical arguments and numerical experiments.