mlsanityData to 14 Oct 2026

One Router to Route Them All: Homogeneous Expert Routing for Heterogeneous Graph Transformers

arXiv 2511.07603 ↗First posted 10 Nov 2025cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

A common practice in heterogeneous graph neural networks (HGNNs) is to condition parameters on node/edge types, assuming types reflect semantic roles. However, this can cause overreliance on surface-level labels and impede cross-type knowledge transfer. We explore integrating Mixture-of-Experts (MoE) into HGNNs--a direction underexplored despite MoE's success in homogeneous settings. Crucially, we question the need for type-specific experts. We propose Homogeneous Expert Routing (HER), an MoE layer for Heterogeneous Graph Transformers (HGT) that stochastically masks type embeddings during routing to encourage type-agnostic specialization. Evaluated on IMDB, ACM, and DBLP for link prediction, HER consistently outperforms standard HGT and a type-separated MoE baseline. Analysis on IMDB shows HER experts specialize by semantic patterns (e.g., movie genres) rather than node types, confirming routing is driven by latent semantics. Our work demonstrates that regularizing type dependence in expert routing yields more generalizable, efficient, and interpretable representations--a new design principle for heterogeneous graph learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 7 of its 18 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

7 of 18 references.

References · 7

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  4. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  5. 2019Graph Neural Networks for Social Recommendationcited by 714
  6. 2020Heterogeneous Graph Transformercited by 582
  7. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 193

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts
  2. 2025GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
  3. 2025Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
  4. 2025Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
  5. 2026L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
  6. 2026RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing
  7. 2025Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance
  8. 2024Graph Knowledge Distillation to Mixture of Experts