mlsanityData to 14 Oct 2026

CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts

arXiv 2606.13024 ↗First posted 11 Jun 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Granger Causal Discovery (GCD) is fundamental for analyzing temporal dependencies in complex systems. However, existing neural GCD methods predominantly rely on a "one-size-fits-all" paradigm, struggling to capture distribution shifts and dynamic regime changes inherent in real-world time series. This often leads to entangled representations and spurious causal graphs. In this paper, we propose CausalMoE, a billion-scale multimodal Granger causal foundation model that explicitly models patch-level heterogeneity. CausalMoE introduces a Pattern-Routed Mixture of Heterogeneous Experts, which dynamically identifies latent temporal patterns and routes patches to specialized domain experts, effectively decoupling regime-specific mechanisms from shared dynamics. To ensure interpretable graph recovery, we design a Causality-Aware Self-Attention mechanism operating across variables, yielding sparse Granger causal graphs via proximal optimization. Furthermore, CausalMoE is the first to integrate LLMs and VLMs to align numerical signals with textual and visual priors, regularizing causal estimation in complex scenarios. Extensive experiments demonstrate that CausalMoE establishes a new state-of-the-art on fully supervised benchmarks, while effectively generalizing to few-shot settings where traditional methods fail.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

20 of 41 references.

References · 20

  1. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  2. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  3. 2017Detecting causal associations in large nonlinear time series datasetscited by 244
  4. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  5. 2024Large Language Models for Time Series: A Surveycited by 114
  6. 2023Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecastingcited by 93
  7. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  8. 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 58
  9. 2019Economy Statistical Recurrent Units For Inferring Nonlinear Granger Causalitycited by 55
  10. 2023Causal Recurrent Variational Autoencoder for Medical Time Series Generationcited by 45
  11. 2023CUTS+: High-dimensional Causal Discovery from Irregular Time-seriescited by 41
  12. 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 40
8 more
  1. 2022DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptationcited by 36
  2. 2025CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-seriescited by 26
  3. 2024Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shiftcited by 24
  4. 2025FilterTS: Comprehensive Frequency Filtering for Multivariate Time Series Forecastingcited by 16
  5. 2024Jacobian Regularizer-based Neural Granger Causalitycited by 9
  6. 2022Granger causal inference on DAGs identifies genomic loci regulating transcriptioncited by 8
  7. 2025Kolmogorov-Arnold Networks for Time Series Granger Causality Inferencecited by 5
  8. 2024Learning Flexible Time-windowed Granger Causality Integrating Heterogeneous Interventional Time Series Datacited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Causal Discovery to Dynamic Causal Inference in Neural Time Series
  2. 2025Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problem
  3. 2025Context-Specific Causal Graph Discovery with Unobserved Contexts: Non-Stationarity, Regimes and Spatio-Temporal Patterns
  4. 2022Variational Causal Dynamics: Discovering Modular World Models from Interventions
  5. 2026CauScale: Neural Causal Discovery at Scale
  6. 2024Large Language Models for Constrained-Based Causal Discovery
  7. 2024LLM-initialized Differentiable Causal Discovery
  8. 2025TimeMKG: Knowledge-Infused Causal Reasoning for Multivariate Time Series Modeling