mlsanityData to 14 Oct 2026

Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modeling

arXiv 2605.27286 ↗First posted 26 May 2026cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Time series foundation models (TSFMs) are transforming the forecasting paradigm through large-scale cross-domain pretraining. However, most existing TSFMs remain univariate, and recent efforts to enable cross-variate modeling still operate directly within the raw variate space. This design introduces fundamental limitations in semantic alignment and relational expressivity. Specifically, raw-space group mixing lacks a dedicated mechanism to align heterogeneous physical quantities, while standard non-negative attention fails to capture the complex synergistic and antagonistic interactions ubiquitous in real-world systems. To address these challenges, we propose Falcon-X, decouples variates from the raw space and maps them into a unified latent prototype space. Falcon-X employs a Unified Prototype Diff-Attention mechanism that explicitly evaluates both positive and negative semantic affinities to explicitly align heterogeneous variates. Cross-variate interactions are then efficiently performed within this shared space via Latent Entity Attention, naturally facilitating zero-shot structural transfer. Finally, a Variate Reassembly Router robustly reconstructs variate-specific trajectories via a request-and-dispatch mechanism. Extensive evaluations on the GIFT-Eval and fev-bench benchmarks demonstrate that Falcon-X achieves excellent forecasting performance, offering a principled and scalable paradigm for complex multivariate environments. Falcon-X is publicly released to support future research.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 33 of its 62 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

33 of 62 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  6. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  7. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  8. 2024Chronos: Learning the Language of Time Seriescited by 759
  9. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  10. 2023A decoder-only foundation model for time-series forecastingcited by 692
  11. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  12. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
21 more
  1. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  2. 2021Monash Time Series Forecasting Archivecited by 267
  3. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  4. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  5. 2025Chronos-2: From Univariate to Universal Forecastingcited by 194
  6. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  7. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
  8. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 69
  9. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 68
  10. 2025fev-bench: A Realistic Benchmark for Time Series Forecastingcited by 64
  11. 2025From Tables to Time: Extending TabPFN-v2 to Time Series Forecastingcited by 60
  12. 2026Toto 2.0: Time Series Forecasting Enters the Scaling Eracited by 30
  13. 2020A machine learning approach for forecasting hierarchical time seriescited by 29
  14. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 28
  15. 2025Foundation Models for Time Series: A Surveycited by 23
  16. 2022SDWPF: A Dataset for Spatial Dynamic Wind Power Forecasting Challenge at KDD Cup 2022cited by 21
  17. 2023LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Predictioncited by 20
  18. 2023AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Datacited by 17
  19. 2026QuitoBench: A High-Quality Open Time Series Forecasting Benchmarkcited by 10
  20. 2022HERMES: Hybrid Error-corrector Model with inclusion of External Signals for nonstationary fashion time seriescited by 5
  21. 2026CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecastingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.