# GraphRARE: Reinforcement Learning Enhanced Graph Neural Network with Relative Entropy

arXiv: https://arxiv.org/abs/2312.09708 · HTML: https://mlsanity.com/paper/2312.09708/

## Key facts

- Cited by 9 papers in the map (data through 7 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 30 of 32)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing](https://mlsanity.com/paper/1905.00067/) (2019)
- [Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks](https://mlsanity.com/paper/2102.06462/) (2021)
- [Graph Neural Networks for Graphs with Heterophily: A Survey](https://mlsanity.com/paper/2202.07082/) (2022)
- [Node Similarity Preserving Graph Convolutional Networks](https://mlsanity.com/paper/2011.09643/) (2020)
- [Non-Local Graph Neural Networks](https://mlsanity.com/paper/2005.14612/) (2020)
- [Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions](https://mlsanity.com/paper/1802.04730/) (2018)
- [GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily](https://mlsanity.com/paper/2110.15777/) (2021)
- [Interpretable and Efficient Heterogeneous Graph Convolutional Network](https://mlsanity.com/paper/2005.13183/) (2020)
- [Search to aggregate neighborhood for graph neural network](https://mlsanity.com/paper/2104.06608/) (2021)
- [Block Modeling-Guided Graph Convolutional Neural Networks](https://mlsanity.com/paper/2112.13507/) (2021)
- [AdaProp: Learning Adaptive Propagation for Graph Neural Network based Knowledge Graph Reasoning](https://mlsanity.com/paper/2205.15319/) (2022)
- [Graph Pointer Neural Networks](https://mlsanity.com/paper/2110.00973/) (2021)
- [Emerging Drug Interaction Prediction Enabled by Flow-based Graph Neural Network with Biomedical Network](https://mlsanity.com/paper/2311.09261/) (2023)
- [IntentGC: a Scalable Graph Convolution Framework Fusing Heterogeneous Information for Recommendation](https://mlsanity.com/paper/1907.12377/) (2019)
- [Using Fuzzy Logic to Leverage HTML Markup for Web Page Representation](https://mlsanity.com/paper/1606.04429/) (2016)
- [Meta-Weight Graph Neural Network: Push the Limits Beyond Global Homophily](https://mlsanity.com/paper/2203.10280/) (2022)
- [Graph Entropy Guided Node Embedding Dimension Selection for Graph Neural Networks](https://mlsanity.com/paper/2105.03178/) (2021)
- [MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoning](https://mlsanity.com/paper/2209.01416/) (2022)
- [Towards Open Temporal Graph Neural Networks](https://mlsanity.com/paper/2303.15015/) (2023)
- [Disconnected Emerging Knowledge Graph Oriented Inductive Link Prediction](https://mlsanity.com/paper/2209.01397/) (2022)
- [AutoAC: Towards Automated Attribute Completion for Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2301.03049/) (2023)

## Built on by (most cited first, 9 of 9)

- [An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training](https://mlsanity.com/paper/2312.11819/) (2023)
- [Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks](https://mlsanity.com/paper/2407.08164/) (2024)
- [GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning](https://mlsanity.com/paper/2505.02027/) (2025)
- [SIGMA: Sheaf-Informed Geometric Multi-Agent Pathfinding](https://mlsanity.com/paper/2502.06440/) (2025)
- [Towards Fair Graph Neural Networks via Graph Counterfactual without Sensitive Attributes](https://mlsanity.com/paper/2412.09947/) (2024)
- [ABG-NAS: Adaptive Bayesian Genetic Neural Architecture Search for Graph Representation Learning](https://mlsanity.com/paper/2504.21254/) (2025)
- [Spatial-Temporal Transformer with Curriculum Learning for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2507.14698/) (2025)
- [Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)](https://mlsanity.com/paper/2409.16590/) (2024)
- [Generative Pre-trained Ranking Model with Over-parameterization at Web-Scale (Extended Abstract)](https://mlsanity.com/paper/2409.16594/) (2024)

## Abstract (arXiv)

Graph neural networks (GNNs) have shown advantages in graph-based analysis tasks. However, most existing methods have the homogeneity assumption and show poor performance on heterophilic graphs, where the linked nodes have dissimilar features and different class labels, and the semantically related nodes might be multi-hop away. To address this limitation, this paper presents GraphRARE, a general framework built upon node relative entropy and deep reinforcement learning, to strengthen the expressive capability of GNNs. An innovative node relative entropy, which considers node features and structural similarity, is used to measure mutual information between node pairs. In addition, to avoid the sub-optimal solutions caused by mixing useful information and noises of remote nodes, a deep reinforcement learning-based algorithm is developed to optimize the graph topology. This algorithm selects informative nodes and discards noisy nodes based on the defined node relative entropy. Extensive experiments are conducted on seven real-world datasets. The experimental results demonstrate the superiority of GraphRARE in node classification and its capability to optimize the original graph topology.