# Transferable Graph Metanetworks

arXiv: https://arxiv.org/abs/2610.00420 · HTML: https://mlsanity.com/paper/2610.00420/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 50 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Predicting Neural Network Accuracy from Weights](https://mlsanity.com/paper/2002.11448/) (2020)
- [Equivariant Deep Weight Space Alignment](https://mlsanity.com/paper/2310.13397/) (2023)
- [A Survey of Weight Space Learning: Understanding, Representation, and Generation](https://mlsanity.com/paper/2603.10090/) (2026)
- [Learning Useful Representations of Recurrent Neural Network Weight Matrices](https://mlsanity.com/paper/2403.11998/) (2024)
- [On Transferring Transferability: Towards a Theory for Size Generalization](https://mlsanity.com/paper/2505.23599/) (2025)
- [A Graph Meta-Network for Learning on Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2602.16316/) (2026)
- [What Linear Probes Miss: Multi-View Probing for Weight-Space Learning](https://mlsanity.com/paper/2605.23410/) (2026)

## Abstract (arXiv)

A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of one or a few fixed sizes and evaluates them in-distribution. The few attempts at out-of-distribution size generalization remain limited in scope and have achieved only modest success. Consequently, the potential efficiency gains of training on small networks and evaluating on much larger ones remain largely unrealized. We propose Transferable Graph Metanetworks, which extend the graph metanetwork paradigm with a set of modifications that make performance transferable across input networks of different widths. The modifications follow two principles: invariance to the ways in which networks of different widths represent the same function, and continuity, such that weights representing similar functions receive similar predictions. We further study whether size generalization is possible for input networks trained independently from random initialization. Empirically, our modifications significantly improve size generalization on every task we consider. Performance is strongest on input networks trained under the maximal-update parameterization (μP), where it remains robust up to 42× the training width. Theoretically, we explain these observations with infinite-width limit theory: we prove size-generalization guarantees for our model on μP-trained inputs, and explain why it can fail under other parameterizations.