# Optimizing the Training Diet: Data Mixture Search for Robust Time Series Forecasting

arXiv: https://arxiv.org/abs/2512.11546 · HTML: https://mlsanity.com/paper/2512.11546/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Builds on (most cited first, 11 of 11)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Balanced Data Sampling for Language Model Training with Clustering](https://mlsanity.com/paper/2402.14526/) (2024)
- [T-Rep: Representation Learning for Time Series using Time-Embeddings](https://mlsanity.com/paper/2310.04486/) (2023)
- [Curriculum Learning in Deep Neural Networks for Financial Forecasting](https://mlsanity.com/paper/1904.12887/) (2019)
- [Time Series Embedding Methods for Classification Tasks: A Review](https://mlsanity.com/paper/2501.13392/) (2025)
- [Curriculum Learning for Deep Generative Models with Clustering](https://mlsanity.com/paper/1906.11594/) (2019)
- [Review of Data-centric Time Series Analysis from Sample, Feature, and Period](https://mlsanity.com/paper/2404.16886/) (2024)

## Abstract (arXiv)

The standard paradigm for training deep learning models on sensor data assumes that more data is always better. However, raw sensor streams are often imbalanced and contain significant redundancy, meaning that not all data points contribute equally to model generalization. In this paper, we show that, in some cases, "less is more" when considering datasets. We do this by reframing the data selection problem: rather than tuning model hyperparameters, we fix the model and optimize the composition of the training data itself. We introduce a framework for discovering the optimal "training diet" from a large, unlabeled time series corpus. Our framework first uses a large-scale encoder and k-means clustering to partition the dataset into distinct, behaviorally consistent clusters. These clusters represent the fundamental 'ingredients' available for training. We then employ the Optuna optimization framework to search the high-dimensional space of possible data mixtures. For each trial, Optuna proposes a specific sampling ratio for each cluster, and a new training set is constructed based on this recipe. A smaller target model is then trained and evaluated. Our experiments reveal that this data-centric search consistently discovers data mixtures that yield models with significantly higher performance compared to baselines trained on the entire dataset. Specifically - evaluated on PMSM dataset - our method improved performance from a baseline MSE of 1.70 to 1.37, a 19.41% improvement.