Post-Training Neural Network Pruning using Graph Curvature
Not on the map. Why?
Abstract (arXiv)
This paper provides a fresh view of the neural network (NN) pruning problem through the lens of graph theory. To achieve effective pruning, we aim to identify the main NN data flows and the corresponding NN connections that are most and least important for the performance of the full model. Unlike the standard approach to NN data flow analysis, which is based on information theory, we employ the notion of graph curvature, specifically Ollivier-Ricci curvature (ORC). ORC has been successfully used to identify important graph edges in various domains such as road traffic analysis, biological networks, and social networks. In particular, edges with negative ORC are considered bottlenecks and are therefore critical to the graph's overall connectivity, whereas positive-ORC edges are less essential. We use this intuition for NNs to (1) construct a graph induced by the NN structure and introduce the notion of neural curvature (NC) based on ORC; (2) calculate curvatures based on activation patterns for a set of input examples; and (3) demonstrate that NC can be used to rank edges according to their importance for overall NN functionality. We evaluate our method through pruning experiments on a variety of small and medium size models trained on three image datasets: MNIST, CIFAR-10, and CIFAR-100. The results indicate that our method can identify a larger number of unimportant edges compared to existing pruning methods.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 22 Jan 2026.
Builds on
27 of 52 references.
References · 27
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023GPT-4 Technical Reportcited by 18,132
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
- 2015Deep Learning and the Information Bottleneck Principlecited by 793
- 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
- 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
- 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
15 more
- 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
- 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
- 2019Community Detection on Networks with Ricci Flowcited by 62
- 2018Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distancecited by 56
- 2024Everybody Prune Now: Structured Pruning of LLMs with only Forward Passescited by 42
- 2019Where is the Information in a Deep Neural Network?cited by 40
- 2021A Self-supervised Mixed-curvature Graph Neural Networkcited by 32
- 2023What Matters In The Structured Pruning of Generative Language Models?cited by 30
- 2015Ricci Curvature of the Internet Topologycited by 22
- 2024No Free Prune: Information-Theoretic Barriers to Pruning at Initializationcited by 8
- 2024Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Modelscited by 7
- 2018Measuring Road Network Topology Vulnerability by Ricci Curvaturecited by 5
- 2016Flow of Information in Feed-Forward Deep Neural Networkscited by 3
- 2024Analyzing Neural Network Robustness Using Graph Curvaturecited by 2
- 2017Modeling Information Flow Through Deep Neural Networkscited by 1
Built on by
Not cited yet.