mlsanityData to 7 Oct 2026

Optimization of Graph Neural Networks with Natural Gradient Descent

arXiv 2008.09624 ↗First posted 21 Aug 2020cs.LG stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

In this work, we propose to employ information-geometric tools to optimize a graph neural network architecture such as the graph convolutional networks. More specifically, we develop optimization algorithms for the graph-based semi-supervised learning by employing the natural gradient information in the optimization process. This allows us to efficiently exploit the geometry of the underlying statistical model or parameter space for optimization and inference. To the best of our knowledge, this is the first work that has utilized the natural gradient for the optimization of graph neural networks that can be extended to other semi-supervised problems. Efficient computations algorithms are developed and extensive numerical studies are conducted to demonstrate the superior performance of our algorithms over existing algorithms such as ADAM and SGD.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 3; 2023: 6; 2024: 7; 2025: 3; 2026: 1.

26 papers.

Sort:

  1. 2021On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphscited by 49
  2. 2023Kronecker-Factored Approximate Curvature for Modern Neural Network Architecturescited by 49
  3. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33
  4. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  5. 2023ASDL: A Unified Interface for Gradient Preconditioning in PyTorchcited by 20
  6. 2022Geometry and convergence of natural policy gradient methodscited by 12
  7. 2021GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graphcited by 11
  8. 2023Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFACcited by 9
  9. 2024LOGIN: A Large Language Model Consulted Graph Neural Network Training Frameworkcited by 8
  10. 2024FSW-GNN: A Bi-Lipschitz WL-Equivalent Graph Neural Networkcited by 8
  11. 2023Structural Node Embeddings with Homomorphism Countscited by 3
  12. 2023Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradientscited by 3
14 more
  1. 2021Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networkscited by 2
  2. 2024Beyond the Known: Novel Class Discovery for Open-world Graph Learningcited by 2
  3. 2024A Benchmark for Fairness-Aware Graph Learningcited by 2
  4. 2024Graph Neural Networks Need Cluster-Normalize-Activate Modulescited by 2
  5. 2021An Energy-Based View of Graph Neural Networkscited by 1
  6. 2023Evaluating Neighbor Explainability for Graph Neural Networkscited by 1
  7. 2021Quadratic GCN for Graph Classificationcited by 0
  8. 2021Affinity Mixup for Weakly Supervised Sound Event Detectioncited by 0
  9. 2022Extrinsic Bayesian Optimizations on Manifoldscited by 0
  10. 2024GraphViz2Vec: A Structure-aware Feature Generation Model to Improve Classification in GNNscited by 0
  11. 2025ReFill: Reinforcement Learning for Fill-In Minimizationcited by 0
  12. 2025Conditional Temporal Neural Processes with Covariance Losscited by 0
  13. 2025Visualization and Analysis of the Loss Landscape in Graph Neural Networkscited by 0
  14. 2026Multi-Scale Temporal Homeostasis Enables Efficient and Robust Neural Networkscited by 0