# Improving Time Series Classification with Representation Soft Label Smoothing

arXiv: https://arxiv.org/abs/2408.17010 · HTML: https://mlsanity.com/paper/2408.17010/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 30 Aug 2024.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Delving Deep into Label Smoothing](https://mlsanity.com/paper/2011.12562/) (2020)
- [Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approach](https://mlsanity.com/paper/2007.09712/) (2020)
- [DisturbLabel: Regularizing CNN on the Loss Layer](https://mlsanity.com/paper/1605.00055/) (2016)
- [Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing](https://mlsanity.com/paper/2005.00820/) (2020)
- [Zero-Shot Knowledge Distillation from a Decision-Based Black-Box Model](https://mlsanity.com/paper/2106.03310/) (2021)
- [TEASER: Early and Accurate Time Series Classification](https://mlsanity.com/paper/1908.03405/) (2019)
- [Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression](https://mlsanity.com/paper/2104.03164/) (2021)
- [KDCTime: Knowledge Distillation with Calibration on InceptionTime for Time-series Classification](https://mlsanity.com/paper/2112.02291/) (2021)
- [TC-DTW: Accelerating Multivariate Dynamic Time Warping Through Triangle Inequality and Point Clustering](https://mlsanity.com/paper/2101.07731/) (2021)

## Abstract (arXiv)

Previous research has indicated that deep neural network based models for time series classification (TSC) tasks are prone to overfitting. This issue can be mitigated by employing strategies that prevent the model from becoming overly confident in its predictions, such as label smoothing and confidence penalty. Building upon the concept of label smoothing, we propose a novel approach to generate more reliable soft labels, which we refer to as representation soft label smoothing. We apply label smoothing, confidence penalty, and our method representation soft label smoothing to several TSC models and compare their performance with baseline method which only uses hard labels for training. Our results demonstrate that the use of these enhancement techniques yields competitive results compared to the baseline method. Importantly, our method demonstrates strong performance across models with varying structures and complexities.