# Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modeling

arXiv: https://arxiv.org/abs/2605.27286 · HTML: https://mlsanity.com/paper/2605.27286/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 62 references that are in the map.
- First posted on arXiv 26 May 2026.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [Chronos-2: From Univariate to Universal Forecasting](https://mlsanity.com/paper/2510.15821/) (2025)
- [Sundial: A Family of Highly Capable Time Series Foundation Models](https://mlsanity.com/paper/2502.00816/) (2025)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [Moirai 2.0: When Less Is More for Time Series Forecasting](https://mlsanity.com/paper/2511.11698/) (2025)
- [This Time is Different: An Observability Perspective on Time Series Foundation Models](https://mlsanity.com/paper/2505.14766/) (2025)
- [fev-bench: A Realistic Benchmark for Time Series Forecasting](https://mlsanity.com/paper/2509.26468/) (2025)
- [From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting](https://mlsanity.com/paper/2501.02945/) (2025)
- [Toto 2.0: Time Series Forecasting Enters the Scaling Era](https://mlsanity.com/paper/2605.20119/) (2026)
- [A machine learning approach for forecasting hierarchical time series](https://mlsanity.com/paper/2006.00630/) (2020)
- [Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling](https://mlsanity.com/paper/2603.04791/) (2026)
- [Foundation Models for Time Series: A Survey](https://mlsanity.com/paper/2504.04011/) (2025)
- [SDWPF: A Dataset for Spatial Dynamic Wind Power Forecasting Challenge at KDD Cup 2022](https://mlsanity.com/paper/2208.04360/) (2022)
- [LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction](https://mlsanity.com/paper/2304.14343/) (2023)
- [AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Data](https://mlsanity.com/paper/2310.20280/) (2023)

## Built on by (most cited first, 10 of 10)

- [Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models](https://mlsanity.com/paper/2606.14222/) (2026)
- [Into the ORBIT for Time Series: Training Regimes for Foundation Models](https://mlsanity.com/paper/2608.13262/) (2026)
- [t0: A Time-Series Foundation Model for Forecasting with Context](https://mlsanity.com/paper/2609.24559/) (2026)
- [Existence Precedes Value: Joint Modeling of Observational Existence and Evolving States in Time Series Forecasting](https://mlsanity.com/paper/2606.13571/) (2026)
- [Navigating the Safety-Fidelity Trade-off: Massive-Variate Time Series Forecasting for Power Systems via Probabilistic Scenarios](https://mlsanity.com/paper/2606.13338/) (2026)
- [When Tomorrow Becomes Today: Self-Evolving Policies for Agentic Time-Series Forecasting](https://mlsanity.com/paper/2609.24862/) (2026)
- [Aurora-X: Built for Extreme Time Series Forecasting](https://mlsanity.com/paper/2609.31038/) (2026)
- [Pythia: Toward Foundation World Models for Multimodal Time Series](https://mlsanity.com/paper/2610.05240/) (2026)
- [A Unified Scaling Law for Time Series Foundation Models](https://mlsanity.com/paper/2610.05269/) (2026)
- [Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives](https://mlsanity.com/paper/2610.11734/) (2026)

## Abstract (arXiv)

Time series foundation models (TSFMs) are transforming the forecasting paradigm through large-scale cross-domain pretraining. However, most existing TSFMs remain univariate, and recent efforts to enable cross-variate modeling still operate directly within the raw variate space. This design introduces fundamental limitations in semantic alignment and relational expressivity. Specifically, raw-space group mixing lacks a dedicated mechanism to align heterogeneous physical quantities, while standard non-negative attention fails to capture the complex synergistic and antagonistic interactions ubiquitous in real-world systems. To address these challenges, we propose Falcon-X, decouples variates from the raw space and maps them into a unified latent prototype space. Falcon-X employs a Unified Prototype Diff-Attention mechanism that explicitly evaluates both positive and negative semantic affinities to explicitly align heterogeneous variates. Cross-variate interactions are then efficiently performed within this shared space via Latent Entity Attention, naturally facilitating zero-shot structural transfer. Finally, a Variate Reassembly Router robustly reconstructs variate-specific trajectories via a request-and-dispatch mechanism. Extensive evaluations on the GIFT-Eval and fev-bench benchmarks demonstrate that Falcon-X achieves excellent forecasting performance, offering a principled and scalable paradigm for complex multivariate environments. Falcon-X is publicly released to support future research.