mlsanityData to 7 Oct 2026

Graph Structure of Neural Networks

arXiv 2007.06559 ↗First posted 13 Jul 2020cs.LG cs.CV cs.SIcited by 67

Not on the map. Why?

Abstract (arXiv)

Neural networks are often represented as graphs of connections between neurons. However, despite their wide use, there is currently little understanding of the relationship between the graph structure of the neural network and its predictive performance. Here we systematically investigate how does the graph structure of neural networks affect their predictive performance. To this end, we develop a novel graph-based representation of neural networks called relational graph, where layers of neural network computation correspond to rounds of message exchange along the graph structure. Using this representation we show that: (1) a "sweet spot" of relational graphs leads to neural networks with significantly improved predictive performance; (2) neural network's performance is approximately a smooth function of the clustering coefficient and average path length of its relational graph; (3) our findings are consistent across many different tasks and datasets; (4) the sweet spot can be identified efficiently; (5) top-performing neural networks have graph structure surprisingly similar to those of real biological neural networks. Our work opens new directions for the design of neural architectures and the understanding on neural networks in general.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 14; 2023: 11; 2024: 7; 2025: 12; 2026: 1.

67 papers.

Sort:

By area:

  1. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
  2. 2021Identity-aware Graph Neural Networkscited by 197
  3. 2021Graph Neural Networks in Network Neurosciencecited by 91
  4. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  5. 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 55
  6. 2023When Brain-inspired AI Meets AGIcited by 54
  7. 2021Automated Machine Learning on Graphs: A Surveycited by 38
  8. 2022Temporal Domain Generalization with Drift-Aware Dynamic Neural Networkscited by 32
  9. 2021Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERTcited by 31
  10. 2021Understanding the Dynamics of DNNs Using Graph Modularitycited by 26
  11. 2021The Elastic Lottery Ticket Hypothesiscited by 23
  12. 2024Scale Equivariant Graph Metanetworkscited by 23
38 more
  1. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
  2. 2021Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspectivecited by 19
  3. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  4. 2021Exploring Robust Architectures for Deep Artificial Neural Networkscited by 11
  5. 2021Quantifying Local Specialization in Deep Neural Networkscited by 10
  6. 2022Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directionscited by 8
  7. 2024Optimal Recurrent Network Topologies for Dynamical Systems Reconstructioncited by 8
  8. 2021Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metricscited by 7
  9. 2021Graph-Based Similarity of Neural Network Representationscited by 7
  10. 2022A Unified and Biologically-Plausible Relational Graph Representation of Vision Transformerscited by 7
  11. 2022Improving Deep Neural Network Random Initialization Through Neuronal Rewiringcited by 7
  12. 2023Core-Periphery Principle Guided Redesign of Self-Attention in Transformerscited by 7
  13. 2023Building Extraction from Remote Sensing Images via an Uncertainty-Aware Networkcited by 7
  14. 2023NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learningcited by 6
  15. 2021Graph-based Neural Architecture Search with Operation Embeddingscited by 5
  16. 2022GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Modelscited by 5
  17. 2023Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networkscited by 5
  18. 2022Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysiscited by 4
  19. 2022Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecturecited by 4
  20. 2022Coupling Visual Semantics of Artificial Neural Networks and Human Brain Function via Synchronized Activationscited by 4
  21. 2022Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networkscited by 4
  22. 2024Principled Architecture-aware Scaling of Hyperparameterscited by 4
  23. 2021RAN-GNNs: breaking the capacity limits of graph neural networkscited by 3
  24. 2023Beyond Multilayer Perceptrons: Investigating Complex Topologies in Neural Networkscited by 3
  25. 2023Understanding Sparse Neural Networks from their Topology via Multipartite Graph Representationscited by 2
  26. 2025NN-Former: Rethinking Graph Structure in Neural Architecture Representationcited by 2
  27. 2025Task complexity shapes internal representations and robustness in neural networkscited by 2
  28. 2021GGT: Graph-Guided Testing for Adversarial Sample Detection of Deep Neural Networkcited by 1
  29. 2021RGP: Neural Network Pruning through Its Regular Graph Structurecited by 1
  30. 2021Leveraging the Graph Structure of Neural Network Training Dynamicscited by 1
  31. 2024Geometric sparsification in recurrent neural networkscited by 1
  32. 2025Semantic Depth Matters: Explaining Errors of Deep Vision Networks through Perceived Class Similaritiescited by 1
  33. 2025Structure-Aware Automatic Channel Pruning by Searching with Graph Embeddingcited by 1
  34. 2025Effective Fine-Tuning with Eigenvector Centrality Based Pruningcited by 1
  35. 2021Livewired Neural Networks: Making Neurons That Fire Together Wire Togethercited by 0
  36. 2021Generalization of graph network inferences in higher-order graphical modelscited by 0
  37. 2021Subspace Graph Physics: Real-Time Rigid Body-Driven Granular Flow Simulationcited by 0
  38. 2021Network Graph Based Neural Architecture Searchcited by 0

50 most cited of 67 All 67 (JSON)