# THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model

arXiv: https://arxiv.org/abs/2607.27303 · HTML: https://mlsanity.com/paper/2607.27303/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding](https://mlsanity.com/paper/2002.01680/) (2020)
- [Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks](https://mlsanity.com/paper/2112.14936/) (2021)
- [An Attention-based Graph Neural Network for Heterogeneous Structural Learning](https://mlsanity.com/paper/1912.10832/) (2019)
- [Simple and Efficient Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2207.02547/) (2022)
- [Interpretable and Efficient Heterogeneous Graph Convolutional Network](https://mlsanity.com/paper/2005.13183/) (2020)
- [Heterogeneous Graph Representation Learning with Relation Awareness](https://mlsanity.com/paper/2105.11122/) (2021)
- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Heterogeneous Temporal Graph Neural Network](https://mlsanity.com/paper/2110.13889/) (2021)
- [Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domains](https://mlsanity.com/paper/2412.08937/) (2024)
- [Billion-Scale Graph Foundation Models](https://mlsanity.com/paper/2602.04768/) (2026)
- [CTRL: Continuous-Time Representation Learning on Temporal Heterogeneous Information Network](https://mlsanity.com/paper/2405.08013/) (2024)
- [Heta: Distributed Training of Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2408.09697/) (2024)

## Abstract (arXiv)

Temporal heterogeneous graphs offer a natural abstraction for dynamic relational systems in which diverse node and relation types co-exist and evolve over time. Learning on such graphs requires jointly modeling cross-type structural heterogeneity and the temporal dynamics of interactions, yet existing methods still struggle to reconcile parameter-efficient cross-type transfer with relation-aware specialization, and typically inject time only as additive features outside the attention kernel. We propose THGFM, a web-scale temporal heterogeneous graph fusion model that addresses both limitations within a unified dual-path architecture. THGFM couples a textitShared-Space Temporal Attention branch for parameter-efficient cross-type transfer with a textitRelational Type-Partitioned Temporal Attention branch for relation-aware specialization, and integrates them through textitDual-Path Relational--Shared Fusion, instantiated with textitType-Conditioned Non-Competitive Gated Sum Fusion: a adaptive mechanism that assigns independent, type-conditioned feature-wise gates to the shared and specialized branches, allowing both to be amplified or suppressed without zero-sum competition. To directly incorporate relative time into the attention score, THGFM further introduces textitRotary Temporal Attention, which rotates queries and keys by half-phases of relative time before matching. THGFM consistently outperforms baseline graph transformer models on academic graphs benchmarks, delivering a +3.25% six-task mean gain, with peak relative gains of +12.37% on OAG-CS PV, +4.87% on PF-L2, and +1.18% on PF-L1, and +4.24%, +3.73%, and +4.61% on OGBN-MAG, HTAG-ArXiv, and HTAG-DBLP, respectively.