# Graph Entropy Guided Node Embedding Dimension Selection for Graph Neural Networks

arXiv: https://arxiv.org/abs/2105.03178 · HTML: https://mlsanity.com/paper/2105.03178/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 7 May 2021.

## Builds on (most cited first, 10 of 10)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Hyperbolic Graph Convolutional Neural Networks](https://mlsanity.com/paper/1910.12933/) (2019)
- [SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism](https://mlsanity.com/paper/2101.08170/) (2021)
- [On the Dimensionality of Word Embedding](https://mlsanity.com/paper/1812.04224/) (2018)
- [Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification](https://mlsanity.com/paper/1906.04898/) (2019)
- [Single Training Dimension Selection for Word Embedding with PCA](https://mlsanity.com/paper/1909.01761/) (2019)
- [Word2vec Skip-gram Dimensionality Selection via Sequential Normalized Maximum Likelihood](https://mlsanity.com/paper/2008.07720/) (2020)

## Built on by (most cited first, 13 of 13)

- [Subgraph Federated Learning with Missing Neighbor Generation](https://mlsanity.com/paper/2106.13430/) (2021)
- [On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs](https://mlsanity.com/paper/2107.01495/) (2021)
- [SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization](https://mlsanity.com/paper/2303.09778/) (2023)
- [GraphRARE: Reinforcement Learning Enhanced Graph Neural Network with Relative Entropy](https://mlsanity.com/paper/2312.09708/) (2023)
- [Graph Ordering Attention Networks](https://mlsanity.com/paper/2204.05351/) (2022)
- [Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classification](https://mlsanity.com/paper/2302.09755/) (2023)
- [Lambda: Learning Matchable Prior For Entity Alignment with Unlabeled Dangling Cases](https://mlsanity.com/paper/2403.10978/) (2024)
- [Heterogeneous Interaction Modeling With Reduced Accumulated Error for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2410.21342/) (2024)
- [Applications of Entropy in Data Analysis and Machine Learning: A Review](https://mlsanity.com/paper/2503.02921/) (2025)
- [Rank Is Not Capacity: Spectral Occupancy for Latent Graph Models](https://mlsanity.com/paper/2605.11142/) (2026)
- [Hybrid Hypergraph Networks for Multimodal Sequence Data Classification](https://mlsanity.com/paper/2508.00926/) (2025)
- [The Impact of Dimensionality on the Stability of Node Embeddings](https://mlsanity.com/paper/2604.08492/) (2026)
- [In Data or Invisible: Toward a Better Digital Representation of Low-Resource Languages with Knowledge Graphs](https://mlsanity.com/paper/2605.05931/) (2026)

## Abstract (arXiv)

Graph representation learning has achieved great success in many areas, including e-commerce, chemistry, biology, etc. However, the fundamental problem of choosing the appropriate dimension of node embedding for a given graph still remains unsolved. The commonly used strategies for Node Embedding Dimension Selection (NEDS) based on grid search or empirical knowledge suffer from heavy computation and poor model performance. In this paper, we revisit NEDS from the perspective of minimum entropy principle. Subsequently, we propose a novel Minimum Graph Entropy (MinGE) algorithm for NEDS with graph data. To be specific, MinGE considers both feature entropy and structure entropy on graphs, which are carefully designed according to the characteristics of the rich information in them. The feature entropy, which assumes the embeddings of adjacent nodes to be more similar, connects node features and link topology on graphs. The structure entropy takes the normalized degree as basic unit to further measure the higher-order structure of graphs. Based on them, we design MinGE to directly calculate the ideal node embedding dimension for any graph. Finally, comprehensive experiments with popular Graph Neural Networks (GNNs) on benchmark datasets demonstrate the effectiveness and generalizability of our proposed MinGE.