LEED: Local Embedding Evolution Distance for over-smoothing estimation and virtual node selection in GNN
Not on the map. Why?
Abstract (arXiv)
Graph Neural Networks (GNNs) suffer from two fundamental limitations: over-smoothing, where node representations become indistinguishable with depth, and over-squashing, where long-range information is compressed through limited message-passing channels. Existing metrics such as Dirichlet energy provide global characterizations of over-smoothing but lack the resolution to analyze node-level behavior and guide architectural improvements. In this paper, we propose LEED (Local Embedding Evolution Distance), a novel local metric that quantifies over-smoothing by tracking the evolution of individual node embeddings across layers. By operating at the node level, LEED enables fine-grained analysis of representation dynamics during training, revealing heterogeneous over-smoothing patterns that are invisible to global energy-based measures. This locality induces informative node importance scores, interpreted as embedding-driven centrality measures. We leverage LEED to design a more efficient strategy for virtual node selection. Unlike existing approaches that depend on multiple heuristic centrality measures, our method uses LEED as a unique criterion to guide the construction of Local Virtual Nodes to mitigate over-squashing. Experiments show that LEED provides more informative diagnostics than Dirichlet energy while preserving global evaluation, and enables more effective virtual node integration, improving GNN performance across datasets.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
28 of 34 references.
References · 28
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2019Simplifying Graph Convolutional Networkscited by 1,470
- 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
- 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
- 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
- 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
- 2022On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networkscited by 63
- 2023On the Connection Between MPNN and Graph Transformercited by 60
- 2023Virtual Node Tuning for Few-shot Node Classificationcited by 46
- 2023Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvaturecited by 39
- 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
16 more
- 2022Rewiring with Positional Encodings for Graph Neural Networkscited by 35
- 2023Over-Squashing in Graph Neural Networks: A Comprehensive surveycited by 23
- 2024Cayley Graph Propagationcited by 21
- 2024Understanding Virtual Nodes: Oversquashing and Node Heterogeneitycited by 14
- 2023Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalizationcited by 12
- 2024The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisitedcited by 11
- 2024PANDA: Expanded Width-Aware Message Passing Beyond Rewiringcited by 10
- 2024Understanding Oversmoothing in Diffusion-Based GNNs From the Perspective of Operator Semigroup Theorycited by 8
- 2025Fast and Distributed Equivariant Graph Neural Networks by Virtual Node Learningcited by 6
- 2023Fault Detection in Telecom Networks using Bi-level Federated Graph Neural Networkscited by 5
- 2022Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approachescited by 4
- 2024Demystifying Higher-Order Graph Neural Networkscited by 4
- 2023Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignmentcited by 2
- 2023Enhancing 5G Radio Planning with Graph Representations and Deep Learningcited by 1
- 2024Gradient Rewiring for Editable Graph Neural Network Trainingcited by 1
- 2025Local Virtual Nodes for Alleviating Over-Squashing in Graph Neural Networkscited by 1
Built on by
Not cited yet.