# CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts

arXiv: https://arxiv.org/abs/2606.13024 · HTML: https://mlsanity.com/paper/2606.13024/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 20 of 20)

- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Detecting causal associations in large nonlinear time series datasets](https://mlsanity.com/paper/1702.07007/) (2017)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [Large Language Models for Time Series: A Survey](https://mlsanity.com/paper/2402.01801/) (2024)
- [Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting](https://mlsanity.com/paper/2302.14829/) (2023)
- [VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2408.17253/) (2024)
- [TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting](https://mlsanity.com/paper/2410.11802/) (2024)
- [Economy Statistical Recurrent Units For Inferring Nonlinear Granger Causality](https://mlsanity.com/paper/1911.09879/) (2019)
- [Causal Recurrent Variational Autoencoder for Medical Time Series Generation](https://mlsanity.com/paper/2301.06574/) (2023)
- [CUTS+: High-dimensional Causal Discovery from Irregular Time-series](https://mlsanity.com/paper/2305.05890/) (2023)
- [TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster](https://mlsanity.com/paper/2503.07649/) (2025)
- [DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation](https://mlsanity.com/paper/2201.04038/) (2022)
- [CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-series](https://mlsanity.com/paper/2503.17452/) (2025)
- [Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift](https://mlsanity.com/paper/2410.09836/) (2024)
- [FilterTS: Comprehensive Frequency Filtering for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.04158/) (2025)
- [Jacobian Regularizer-based Neural Granger Causality](https://mlsanity.com/paper/2405.08779/) (2024)
- [Granger causal inference on DAGs identifies genomic loci regulating transcription](https://mlsanity.com/paper/2210.10168/) (2022)
- [Kolmogorov-Arnold Networks for Time Series Granger Causality Inference](https://mlsanity.com/paper/2501.08958/) (2025)
- [Learning Flexible Time-windowed Granger Causality Integrating Heterogeneous Interventional Time Series Data](https://mlsanity.com/paper/2406.10419/) (2024)

## Abstract (arXiv)

Granger Causal Discovery (GCD) is fundamental for analyzing temporal dependencies in complex systems. However, existing neural GCD methods predominantly rely on a "one-size-fits-all" paradigm, struggling to capture distribution shifts and dynamic regime changes inherent in real-world time series. This often leads to entangled representations and spurious causal graphs. In this paper, we propose CausalMoE, a billion-scale multimodal Granger causal foundation model that explicitly models patch-level heterogeneity. CausalMoE introduces a Pattern-Routed Mixture of Heterogeneous Experts, which dynamically identifies latent temporal patterns and routes patches to specialized domain experts, effectively decoupling regime-specific mechanisms from shared dynamics. To ensure interpretable graph recovery, we design a Causality-Aware Self-Attention mechanism operating across variables, yielding sparse Granger causal graphs via proximal optimization. Furthermore, CausalMoE is the first to integrate LLMs and VLMs to align numerical signals with textual and visual priors, regularizing causal estimation in complex scenarios. Extensive experiments demonstrate that CausalMoE establishes a new state-of-the-art on fully supervised benchmarks, while effectively generalizing to few-shot settings where traditional methods fail.