DiGRAF: Diffeomorphic Graph-Adaptive Activation Function
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose a novel activation function tailored specifically for graph data in Graph Neural Networks (GNNs). Motivated by the need for graph-adaptive and flexible activation functions, we introduce DiGRAF, leveraging Continuous Piecewise-Affine Based (CPAB) transformations, which we augment with an additional GNN to learn a graph-adaptive diffeomorphic activation function in an end-to-end manner. In addition to its graph-adaptivity and flexibility, DiGRAF also possesses properties that are widely recognized as desirable for activation functions, such as differentiability, boundness within the domain, and computational efficiency. We conduct an extensive set of experiments across diverse datasets and tasks, demonstrating a consistent and superior performance of DiGRAF compared to traditional and graph-specific activation functions, highlighting its effectiveness as an activation function for GNNs. Our code is available at https://github.com/ipsitmantri/DiGRAF.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 57 of its 96 references that are in the map. First posted on arXiv 2 Jul 2024.
Builds on
57 of 96 references.
References · 57
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
- 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
- 2019Strategies for Pre-training Graph Neural Networkscited by 964
- 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
38 more
- 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
- 2020Simple and Deep Graph Convolutional Networkscited by 857
- 2021How Attentive are Graph Attention Networks?cited by 818
- 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
- 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
- 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
- 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
- 2020Benchmarking Graph Neural Networkscited by 672
- 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 515
- 2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
- 2019Graph U-Netscited by 489
- 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
- 2019Self-Attention Graph Poolingcited by 421
- 2017Stable Architectures for Deep Neural Networkscited by 340
- 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
- 2013Maxout Networkscited by 186
- 2022Graph neural networks for materials science and chemistrycited by 181
- 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 153
- 2017Non-convex Optimization for Machine Learningcited by 132
- 2018Activation Functions: Comparison of trends in Practice and Research for Deep Learningcited by 130
- 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 121
- 2023Rethinking the Expressive Power of GNNs via Graph Biconnectivitycited by 119
- 2022Understanding and Extending Subgraph GNNs by Rethinking Their Symmetriescited by 112
- 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
- 2020A survey on modern trainable activation functionscited by 89
- 2019Hierarchical Graph Pooling with Structure Learningcited by 81
- 2021On the approximation of functions by tanh neural networkscited by 81
- 2023A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Testscited by 64
- 2023Equivariant Polynomials for Graph Neural Networkscited by 38
- 2023Graph Positional Encoding via Random Feature Propagationcited by 30
- 2023Deep Operator Network Approximation Rates for Lipschitz Operatorscited by 24
- 2020PANE: scalable and effective attributed network embeddingcited by 23
- 2020Review and Comparison of Commonly Used Activation Functions for Deep Neural Networkscited by 21
- 2024Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural Networkscited by 21
- 2019Towards Interpretable Sparse Graph Representation Learning with Laplacian Poolingcited by 20
- 2023Efficient Subgraph GNNs by Learning Effective Selection Policiescited by 17
- 2019Effect of Activation Functions on the Training of Overparametrized Neural Netscited by 14
- 2022Graph-adaptive Rectified Linear Unit for Graph Neural Networkscited by 12
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.