# Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency

arXiv: https://arxiv.org/abs/2510.19980 · HTML: https://mlsanity.com/paper/2510.19980/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 29 of 29)

- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [LongNet: Scaling Transformers to 1,000,000,000 Tokens](https://mlsanity.com/paper/2307.02486/) (2023)
- [FITS: Modeling Time Series with 10k Parameters](https://mlsanity.com/paper/2307.03756/) (2023)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Frequency-domain MLPs are More Effective Learners in Time Series Forecasting](https://mlsanity.com/paper/2311.06184/) (2023)
- [TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2306.09364/) (2023)
- [Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis](https://mlsanity.com/paper/2310.06119/) (2023)
- [Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting](https://mlsanity.com/paper/2402.05956/) (2024)
- [SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion](https://mlsanity.com/paper/2404.14197/) (2024)
- [SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2308.11200/) (2023)
- [TACTiS: Transformer-Attentional Copulas for Time Series](https://mlsanity.com/paper/2202.03528/) (2022)
- [A Co-training Approach for Noisy Time Series Learning](https://mlsanity.com/paper/2308.12551/) (2023)
- [Denoising-Aware Contrastive Learning for Noisy Time Series](https://mlsanity.com/paper/2406.04627/) (2024)
- [Long Input Sequence Network for Long Time Series Forecasting](https://mlsanity.com/paper/2407.15869/) (2024)
- [Demand Forecasting in Smart Grid Using Long Short-Term Memory](https://mlsanity.com/paper/2107.13653/) (2021)

## Built on by (most cited first, 1 of 1)

- [PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents](https://mlsanity.com/paper/2605.10341/) (2026)

## Abstract (arXiv)

Time series forecasting plays a pivotal role in critical domains such as energy management and financial markets. Although deep learning-based approaches (e.g., MLP, RNN, Transformer) have achieved remarkable progress, the prevailing "long-sequence information gain hypothesis" exhibits inherent limitations. Through systematic experimentation, this study reveals a counterintuitive phenomenon: appropriately truncating historical data can paradoxically enhance prediction accuracy, indicating that existing models learn substantial redundant features (e.g., noise or irrelevant fluctuations) during training, thereby compromising effective signal extraction. Building upon information bottleneck theory, we propose an innovative solution termed Adaptive Masking Loss with Representation Consistency (AMRC), which features two core components: 1) Dynamic masking loss, which adaptively identified highly discriminative temporal segments to guide gradient descent during model training; 2) Representation consistency constraint, which stabilized the mapping relationships among inputs, labels, and predictions. Experimental results demonstrate that AMRC effectively suppresses redundant feature learning while significantly improving model performance. This work not only challenges conventional assumptions in temporal modeling but also provides novel theoretical insights and methodological breakthroughs for developing efficient and robust forecasting models.