# Hypergraph Transformer for Semi-Supervised Classification

arXiv: https://arxiv.org/abs/2312.11385 · HTML: https://mlsanity.com/paper/2312.11385/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [Hypergraph Convolution and Hypergraph Attention](https://mlsanity.com/paper/1901.08150/) (2019)
- [NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification](https://mlsanity.com/paper/2306.08385/) (2023)
- [Graph Inductive Biases in Transformers without Message Passing](https://mlsanity.com/paper/2305.17589/) (2023)
- [UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks](https://mlsanity.com/paper/2105.00956/) (2021)
- [GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoning](https://mlsanity.com/paper/2204.08770/) (2022)
- [You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks](https://mlsanity.com/paper/2106.13264/) (2021)
- [HNHN: Hypergraph Networks with Hyperedge Neurons](https://mlsanity.com/paper/2006.12278/) (2020)
- [What are higher-order networks?](https://mlsanity.com/paper/2104.11329/) (2021)
- [Equivariant Hypergraph Diffusion Neural Operators](https://mlsanity.com/paper/2207.06680/) (2022)
- [Dynamic-Group-Aware Networks for Multi-Agent Trajectory Prediction with Relational Reasoning](https://mlsanity.com/paper/2206.13114/) (2022)
- [Preventing Over-Smoothing for Hypergraph Neural Networks](https://mlsanity.com/paper/2203.17159/) (2022)
- [Hypergraph Transformer: Weakly-supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2204.10448/) (2022)
- [A nonlinear diffusion method for semi-supervised learning on hypergraphs](https://mlsanity.com/paper/2103.14867/) (2021)
- [Learning Hypergraphs From Signals With Dual Smoothness Prior](https://mlsanity.com/paper/2211.01717/) (2022)
- [A Markov Random Field model for Hypergraph-based Machine Learning](https://mlsanity.com/paper/2308.14172/) (2023)

## Built on by (most cited first, 7 of 7)

- [DHG-Bench: A Comprehensive Benchmark for Deep Hypergraph Learning](https://mlsanity.com/paper/2508.12244/) (2025)
- [Enhancing the Utility of Higher-Order Information in Relational Learning](https://mlsanity.com/paper/2502.09570/) (2025)
- [HyperKAN: Hypergraph Representation Learning with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2503.12365/) (2025)
- [Structure-aware Hypergraph Transformer for Diagnosis Prediction in Electronic Health Records](https://mlsanity.com/paper/2508.20500/) (2025)
- [Knowledge Graph and Hypergraph Transformers with Repository-Attention and Journey-Based Role Transport](https://mlsanity.com/paper/2603.03304/) (2026)
- [Learning Multi-Scale Hypergraph for High-Order Brain Connectivity Analysis](https://mlsanity.com/paper/2606.03310/) (2026)
- [Labeled Incidence Structures for Native Transformer Modeling of Text, Knowledge Graphs, and Hypergraphs](https://mlsanity.com/paper/2609.20278/) (2026)

## Abstract (arXiv)

Hypergraphs play a pivotal role in the modelling of data featuring higher-order relations involving more than two entities. Hypergraph neural networks emerge as a powerful tool for processing hypergraph-structured data, delivering remarkable performance across various tasks, e.g., hypergraph node classification. However, these models struggle to capture global structural information due to their reliance on local message passing. To address this challenge, we propose a novel hypergraph learning framework, HyperGraph Transformer (HyperGT). HyperGT uses a Transformer-based neural network architecture to effectively consider global correlations among all nodes and hyperedges. To incorporate local structural information, HyperGT has two distinct designs: i) a positional encoding based on the hypergraph incidence matrix, offering valuable insights into node-node and hyperedge-hyperedge interactions; and ii) a hypergraph structure regularization in the loss function, capturing connectivities between nodes and hyperedges. Through these designs, HyperGT achieves comprehensive hypergraph representation learning by effectively incorporating global interactions while preserving local connectivity patterns. Extensive experiments conducted on real-world hypergraph node classification tasks showcase that HyperGT consistently outperforms existing methods, establishing new state-of-the-art benchmarks. Ablation studies affirm the effectiveness of the individual designs of our model.