mlsanityData to 14 Oct 2026

On the Internal Representations of Graph Metanetworks

arXiv 2503.09120 ↗First posted 12 Mar 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Weight space learning is an emerging paradigm in the deep learning community. The primary goal of weight space learning is to extract informative features from a set of parameters using specially designed neural networks, often referred to as metanetworks. However, it remains unclear how these metanetworks learn solely from parameters. To address this, we take the first step toward understanding representations of metanetworks, specifically graph metanetworks (GMNs), which achieve state-of-the-art results in this field, using centered kernel alignment (CKA). Through various experiments, we reveal that GMNs and general neural networks (textite.g., multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs)) differ in terms of their representation space.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 23 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

19 of 23 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  4. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  6. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  7. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  8. 2020Predicting Neural Network Accuracy from Weightscited by 110
  9. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
  10. 2021Generative Models as Distributions of Functionscited by 89
  11. 2023Permutation Equivariant Neural Functionalscited by 70
  12. 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
7 more
  1. 2024Graph Neural Networks for Learning Equivariant Representations of Neural Networkscited by 59
  2. 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 57
  3. 2024Do Vision and Language Encoders Represent the World Similarly?cited by 44
  4. 2024Improved Generalization of Weight Space Networks via Augmentationscited by 24
  5. 2024Scale Equivariant Graph Metanetworkscited by 24
  6. 2023How to Train Neural Field Representations: A Comprehensive Study and Benchmarkcited by 12
  7. 2024Fast Training of Sinusoidal Neural Fields via Scaling Initializationcited by 11

Built on by

Not cited yet.