# Towards a Unified Generative Model for Scarce Time Series with Domain Experts

arXiv: https://arxiv.org/abs/2606.15172 · HTML: https://mlsanity.com/paper/2606.15172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 13 Jun 2026.

## Builds on (most cited first, 30 of 32)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Diffusion-TS: Interpretable Diffusion for General Time Series Generation](https://mlsanity.com/paper/2403.01742/) (2024)
- [UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting](https://mlsanity.com/paper/2310.09751/) (2023)
- [TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation](https://mlsanity.com/paper/2111.08095/) (2021)
- [LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training](https://mlsanity.com/paper/2406.16554/) (2024)
- [Bake off redux: a review and experimental evaluation of recent time series classification algorithms](https://mlsanity.com/paper/2304.13029/) (2023)
- [Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts](https://mlsanity.com/paper/2410.10469/) (2024)
- [UniTS: A Unified Multi-Task Time Series Model](https://mlsanity.com/paper/2403.00131/) (2024)
- [Scaling Diffusion Transformers to 16 Billion Parameters](https://mlsanity.com/paper/2407.11633/) (2024)
- [Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs](https://mlsanity.com/paper/2310.02619/) (2023)
- [Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series](https://mlsanity.com/paper/2410.19538/) (2024)
- [TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop](https://mlsanity.com/paper/2503.01013/) (2025)
- [Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer](https://mlsanity.com/paper/2405.17478/) (2024)
- [TSGBench: Time Series Generation Benchmark](https://mlsanity.com/paper/2309.03755/) (2023)
- [Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift](https://mlsanity.com/paper/2410.09836/) (2024)
- [TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts](https://mlsanity.com/paper/2501.05403/) (2025)
- [Population Aware Diffusion for Time Series Generation](https://mlsanity.com/paper/2501.00910/) (2025)
- [Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach](https://mlsanity.com/paper/2505.20446/) (2025)
- [Cross-Domain Pre-training with Language Models for Transferable Time Series Representations](https://mlsanity.com/paper/2403.12372/) (2024)
- [EMOPAIN Challenge 2020: Multimodal Pain Evaluation from Facial and Bodily Expressions](https://mlsanity.com/paper/2001.07739/) (2020)
- [ANT: Adaptive Noise Schedule for Time Series Diffusion Models](https://mlsanity.com/paper/2410.14488/) (2024)

## Abstract (arXiv)

Synthesizing realistic time series with generative models has wide-ranging applications in real-world scenarios. Despite recent progress, most existing methods are trained under the assumption of abundant training data, which substantially limits their effectiveness in data-scarce settings. In this paper, we propose TimeMoDE, a novel framework that integrates Diffusion Transformers with Mixture-of-Experts to exploit both domain adaptability and diffusion-stage awareness for time series generation under data scarcity. It is pre-trained on a large-scale collection of multi-domain datasets to extract domain-agnostic temporal representations and domain-specific information benefiting generalization during fine-tuning. We propose Domain Prompts to condition expert assignment for indistinguishable noised tokens, mitigating the limitations of capturing inter-dataset relationships. Moreover, we incorporate diffusion timestep signals to equip the experts with awareness of time series degradation variations, facilitating adaptive calibrate to stage-dependent denoising requirements. Extensive experiments demonstrate that TimeMoDE outperforms existing methods under diverse low-data settings. It establishes an innovative paradigm for advanced time series few-shot generation.