# LightHGNN: Distilling Hypergraph Neural Networks into MLPs for 100× Faster Inference

arXiv: https://arxiv.org/abs/2402.04296 · HTML: https://mlsanity.com/paper/2402.04296/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 20 of 20)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding](https://mlsanity.com/paper/2002.01680/) (2020)
- [A Note on Over-Smoothing for Graph Neural Networks](https://mlsanity.com/paper/2006.13318/) (2020)
- [Hypergraph Convolution and Hypergraph Attention](https://mlsanity.com/paper/1901.08150/) (2019)
- [Graph Information Bottleneck](https://mlsanity.com/paper/2010.12811/) (2020)
- [Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation](https://mlsanity.com/paper/2110.08727/) (2021)
- [UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks](https://mlsanity.com/paper/2105.00956/) (2021)
- [You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks](https://mlsanity.com/paper/2106.13264/) (2021)
- [HNHN: Hypergraph Networks with Hyperedge Neurons](https://mlsanity.com/paper/2006.12278/) (2020)
- [Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework](https://mlsanity.com/paper/2103.02885/) (2021)
- [Hypergraph Contrastive Collaborative Filtering](https://mlsanity.com/paper/2204.12200/) (2022)
- [Equivariant Hypergraph Diffusion Neural Operators](https://mlsanity.com/paper/2207.06680/) (2022)
- [Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs](https://mlsanity.com/paper/2306.05628/) (2023)
- [Preventing Over-Smoothing for Hypergraph Neural Networks](https://mlsanity.com/paper/2203.17159/) (2022)
- [HyGNN: Drug-Drug Interaction Prediction via Hypergraph Neural Network](https://mlsanity.com/paper/2206.12747/) (2022)
- [Deep Hypergraph Structure Learning](https://mlsanity.com/paper/2208.12547/) (2022)

## Built on by (most cited first, 5 of 5)

- [Training-Free Message Passing for Learning on Hypergraphs](https://mlsanity.com/paper/2402.05569/) (2024)
- [Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference](https://mlsanity.com/paper/2411.14035/) (2024)
- [Heterophily-Agnostic Hypergraph Neural Networks with Riemannian Local Exchanger](https://mlsanity.com/paper/2603.00599/) (2026)
- [H3Former: Hypergraph-based Semantic-Aware Aggregation via Hyperbolic Hierarchical Contrastive Loss for Fine-Grained Visual Classification](https://mlsanity.com/paper/2511.10260/) (2025)
- [Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks](https://mlsanity.com/paper/2606.08978/) (2026)

## Abstract (arXiv)

Hypergraph Neural Networks (HGNNs) have recently attracted much attention and exhibited satisfactory performance due to their superiority in high-order correlation modeling. However, it is noticed that the high-order modeling capability of hypergraph also brings increased computation complexity, which hinders its practical industrial deployment. In practice, we find that one key barrier to the efficient deployment of HGNNs is the high-order structural dependencies during inference. In this paper, we propose to bridge the gap between the HGNNs and inference-efficient Multi-Layer Perceptron (MLPs) to eliminate the hypergraph dependency of HGNNs and thus reduce computational complexity as well as improve inference speed. Specifically, we introduce LightHGNN and LightHGNN^+ for fast inference with low complexity. LightHGNN directly distills the knowledge from teacher HGNNs to student MLPs via soft labels, and LightHGNN^+ further explicitly injects reliable high-order correlations into the student MLPs to achieve topology-aware distillation and resistance to over-smoothing. Experiments on eight hypergraph datasets demonstrate that even without hypergraph dependency, the proposed LightHGNNs can still achieve competitive or even better performance than HGNNs and outperform vanilla MLPs by 16.3 on average. Extensive experiments on three graph datasets further show the average best performance of our LightHGNNs compared with all other methods. Experiments on synthetic hypergraphs with 5.5w vertices indicate LightHGNNs can run 100× faster than HGNNs, showcasing their ability for latency-sensitive deployments.