mlsanityData to 14 Oct 2026

Graph Neural Networks for Learning Equivariant Representations of Neural Networks

arXiv 2403.12143 ↗First posted 18 Mar 2024cs.LG cs.AI stat.MLcited by 59

Not on the map. Why?

Abstract (arXiv)

Neural networks that process the parameters of other neural networks find applications in domains as diverse as classifying implicit neural representations, generating neural network weights, and predicting generalization errors. However, existing approaches either overlook the inherent permutation symmetry in the neural network or rely on intricate weight-sharing patterns to achieve equivariance, while ignoring the impact of the network architecture itself. In this work, we propose to represent neural networks as computational graphs of parameters, which allows us to harness powerful graph neural networks and transformers that preserve permutation symmetry. Consequently, our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations, predicting generalization performance, and learning to optimize, while consistently outperforming state-of-the-art methods. The source code is open-sourced at https://github.com/mkofinas/neural-graphs.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

37 of 47 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2016Layer Normalizationcited by 5,669
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  9. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  10. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  11. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  12. 2016Learning to learn by gradient descent by gradient descentcited by 620
25 more
  1. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  2. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  3. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  4. 2019What Can Neural Networks Reason About?cited by 159
  5. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  6. 2017Learned Optimizers that Scale and Generalizecited by 110
  7. 2020Predicting Neural Network Accuracy from Weightscited by 110
  8. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
  9. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  10. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  11. 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 86
  12. 2019GNN-FiLM: Graph Neural Networks with Feature-wise Linear Modulationcited by 75
  13. 2023Permutation Equivariant Neural Functionalscited by 70
  14. 2020Classifying the classifier: dissecting the weight space of neural networkscited by 65
  15. 2022Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weightscited by 64
  16. 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
  17. 2020Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselvescited by 58
  18. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  19. 2023Neural Functional Transformerscited by 52
  20. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  21. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 39
  22. 2022Practical tradeoffs between memory, compute, and performance in learned optimizerscited by 32
  23. 2020Training Stronger Baselines for Learning to Optimizecited by 31
  24. 2022A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biasescited by 30
  25. 2022NeRN -- Learning Neural Representations for Neural Networkscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 27; 2026: 16.

59 papers.

Sort:

By area:

  1. 2024Towards Scalable and Versatile Weight Space Learningcited by 41
  2. 2025Symmetry in Neural Network Parameter Spacescited by 36
  3. 2025We Should Chart an Atlas of All the World's Modelscited by 25
  4. 2026SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Passcited by 25
  5. 2024Scale Equivariant Graph Metanetworkscited by 24
  6. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 22
  7. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  8. 2024Grounding Continuous Representations in Geometry: Equivariant Neural Fieldscited by 19
  9. 2024Monomial Matrix Group Equivariant Neural Functional Networkscited by 17
  10. 2024Deep Linear Probe Generators for Weight Space Learningcited by 17
  11. 2025Generative Modeling of Weights: Generalization or Memorization?cited by 14
  12. 2024Unsupervised Model Tree Heritage Recoverycited by 11
38 more
  1. 2024Learning on Model Weights using Tree Expertscited by 7
  2. 2025End-to-End Implicit Neural Representations for Classificationcited by 7
  3. 2025GradMetaNet: An Equivariant Architecture for Learning on Gradientscited by 7
  4. 2024Connecting NeRFs, Images, and Textcited by 6
  5. 2024LLaNA: Large Language and NeRF Assistantcited by 6
  6. 2024Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networkscited by 6
  7. 2025Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstructioncited by 6
  8. 2024Neural Solver Selection for Combinatorial Optimizationcited by 5
  9. 2025Celo: Training Versatile Learned Optimizers on a Compute Dietcited by 4
  10. 2025Deep Active Inference Agents for Delayed and Long-Horizon Environmentscited by 4
  11. 2025Learning Model Representations Using Publicly Available Model Hubscited by 4
  12. 2025ARC: Anchored Representation Clouds for High-Resolution INR Classificationcited by 3
  13. 2025Flow to Learn: Flow Matching on Neural Network Parameterscited by 3
  14. 2025A Model Zoo of Vision Transformerscited by 3
  15. 2025Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networkscited by 3
  16. 2024Equivariant Machine Learning on Graphs with Nonlinear Spectral Filterscited by 2
  17. 2024Revisiting Multi-Permutation Equivariance through the Lens of Irreducible Representationscited by 2
  18. 2025Improving Learning to Optimize Using Parameter Symmetriescited by 2
  19. 2025Meta Pruning via Graph Metanetworks : A Universal Meta Learning Framework for Network Pruningcited by 2
  20. 2026A Graph Meta-Network for Learning on Kolmogorov-Arnold Networkscited by 2
  21. 2026What Linear Probes Miss: Multi-View Probing for Weight-Space Learningcited by 2
  22. 2025EqCollide: Equivariant and Collision-Aware Deformable Objects Neural Simulatorcited by 1
  23. 2025Flows and Diffusions on the Neural Manifoldcited by 1
  24. 2025Beyond Pixels: Medical Image Quality Assessment with Implicit Neural Representationscited by 1
  25. 2025GNN-based Unified Deep Learningcited by 1
  26. 2025Weight Space Representation Learning via Neural Field Adaptationcited by 1
  27. 2026Discovering Hidden Gems in Model Repositoriescited by 1
  28. 2026Quasi-Equivariant Metanetworkscited by 1
  29. 2026Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Spacecited by 1
  30. 2024Hyper-Representations: Learning from Populations of Neural Networkscited by 0
  31. 2025Weight Space Representation Learning on Diverse NeRF Architecturescited by 0
  32. 2025On the Internal Representations of Graph Metanetworkscited by 0
  33. 2025Scaling LLaNA: Advancing NeRF-Language Understanding Through Large-Scale Trainingcited by 0
  34. 2025Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworkscited by 0
  35. 2025UnifiedFL: A Dynamic Unified Learning Framework for Equitable Federationcited by 0
  36. 2025Symmetry-Aware Graph Metanetwork Autoencoders: Model Merging through Parameter Canonicalizationcited by 0
  37. 2025Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliancecited by 0
  38. 2026Square Superpixel Generation and Representation Learning via Granular Ball Computingcited by 0

50 most cited of 59 All 59 (JSON)