Transfer Entropy in Graph Convolutional Neural Networks
Not on the map. Why?
Abstract (arXiv)
Graph Convolutional Networks (GCN) are Graph Neural Networks where the convolutions are applied over a graph. In contrast to Convolutional Neural Networks, GCN's are designed to perform inference on graphs, where the number of nodes can vary, and the nodes are unordered. In this study, we address two important challenges related to GCNs: i) oversmoothing; and ii) the utilization of node relational properties (i.e., heterophily and homophily). Oversmoothing is the degradation of the discriminative capacity of nodes as a result of repeated aggregations. Heterophily is the tendency for nodes of different classes to connect, whereas homophily is the tendency of similar nodes to connect. We propose a new strategy for addressing these challenges in GCNs based on Transfer Entropy (TE), which measures of the amount of directed transfer of information between two time varying nodes. Our findings indicate that using node heterophily and degree information as a node selection mechanism, along with feature-based TE calculations, enhances accuracy across various GCN models. Our model can be easily modified to improve classification accuracy of a GCN model. As a trade off, this performance boost comes with a significant computational overhead when the TE is computed for many graph nodes.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 28 of its 51 references that are in the map. First posted on arXiv 8 Jun 2024.
Builds on
28 of 51 references.
References · 28
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,318
- 2017Graph Attention Networkscited by 8,888
- 2017Inductive Representation Learning on Large Graphscited by 6,303
- 2018How Powerful are Graph Neural Networks?cited by 4,373
- 2017Neural Message Passing for Quantum Chemistrycited by 3,471
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,911
- 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,101
- 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
- 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 973
- 2020Simple and Deep Graph Convolutional Networkscited by 857
- 2019Multi-scale Attributed Node Embeddingcited by 546
- 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 473
16 more
- 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 337
- 2023A Survey on Oversmoothing in Graph Neural Networkscited by 247
- 2022Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNscited by 210
- 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
- 2022Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyondcited by 60
- 2021Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networkscited by 18
- 2023Higher-order Graph Convolutional Network with Flower-Petals Laplacians on Simplicial Complexescited by 15
- 2020Multivariate Time Series Forecasting with Transfer Entropy Graphcited by 10
- 2020The Power of Graph Convolutional Networks to Distinguish Random Graph Models: Short Versioncited by 6
- 2017Transfer entropy-based feedback improves performance in artificial neural networkscited by 5
- 2021Learning in Feedforward Neural Networks Accelerated by Transfer Entropycited by 5
- 2022Incorporating Heterophily into Graph Neural Networks for Graph Classificationcited by 4
- 2023GCNH: A Simple Method For Representation Learning On Heterophilous Graphscited by 4
- 2024Learning in Convolutional Neural Networks Accelerated by Transfer Entropycited by 2
- 2023On Addressing the Limitations of Graph Neural Networkscited by 1
- 2024Information Plane Analysis Visualization in Deep Learning via Transfer Entropycited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.