# Deep Linear Probe Generators for Weight Space Learning

arXiv: https://arxiv.org/abs/2410.10811 · HTML: https://mlsanity.com/paper/2410.10811/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 50 of its 62 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [Implicit Regularization in Deep Matrix Factorization](https://mlsanity.com/paper/1905.13655/) (2019)
- [ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs](https://mlsanity.com/paper/2311.13600/) (2023)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [Optimizing the Latent Space of Generative Networks](https://mlsanity.com/paper/1707.05776/) (2017)
- [HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion](https://mlsanity.com/paper/2303.17015/) (2023)
- [Predicting Neural Network Accuracy from Weights](https://mlsanity.com/paper/2002.11448/) (2020)
- [Equivariant Architectures for Learning in Deep Weight Spaces](https://mlsanity.com/paper/2301.12780/) (2023)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)
- [Learning to Learn with Generative Models of Neural Network Checkpoints](https://mlsanity.com/paper/2209.12892/) (2022)
- [Permutation Equivariant Neural Functionals](https://mlsanity.com/paper/2302.14040/) (2023)
- [Classifying the classifier: dissecting the weight space of neural networks](https://mlsanity.com/paper/2002.05688/) (2020)
- [Deep Learning on Implicit Neural Representations of Shapes](https://mlsanity.com/paper/2302.05438/) (2023)

## Built on by (most cited first, 17 of 17)

- [We Should Chart an Atlas of All the World's Models](https://mlsanity.com/paper/2503.10633/) (2025)
- [Learning on Model Weights using Tree Experts](https://mlsanity.com/paper/2410.13569/) (2024)
- [GradMetaNet: An Equivariant Architecture for Learning on Gradients](https://mlsanity.com/paper/2507.01649/) (2025)
- [Weight-Space Linear Recurrent Neural Networks](https://mlsanity.com/paper/2506.01153/) (2025)
- [Learning Model Representations Using Publicly Available Model Hubs](https://mlsanity.com/paper/2510.02096/) (2025)
- [A Model Zoo of Vision Transformers](https://mlsanity.com/paper/2504.10231/) (2025)
- [What Linear Probes Miss: Multi-View Probing for Weight-Space Learning](https://mlsanity.com/paper/2605.23410/) (2026)
- [Discovering Hidden Gems in Model Repositories](https://mlsanity.com/paper/2601.22157/) (2026)
- [Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem](https://mlsanity.com/paper/2601.23181/) (2026)
- [Diagnosing Knowledge Conflict in Multimodal Long-Chain Reasoning](https://mlsanity.com/paper/2602.14518/) (2026)
- [Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM](https://mlsanity.com/paper/2604.25119/) (2026)
- [Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance](https://mlsanity.com/paper/2512.15469/) (2025)
- [ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety](https://mlsanity.com/paper/2604.19083/) (2026)
- [Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?](https://mlsanity.com/paper/2605.08281/) (2026)
- [Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability](https://mlsanity.com/paper/2606.04754/) (2026)
- [Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification](https://mlsanity.com/paper/2607.29463/) (2026)
- [Finite Probes Suffice: Identifiability and Universality for Weight-Space Learning](https://mlsanity.com/paper/2609.33901/) (2026)

## Abstract (arXiv)

Weight space learning aims to extract information about a neural network, such as its training dataset or generalization error. Recent approaches learn directly from model weights, but this presents many challenges as weights are high-dimensional and include permutation symmetries between neurons. An alternative approach, Probing, represents a model by passing a set of learned inputs (probes) through the model, and training a predictor on top of the corresponding outputs. Although probing is typically not used as a stand alone approach, our preliminary experiment found that a vanilla probing baseline worked surprisingly well. However, we discover that current probe learning strategies are ineffective. We therefore propose Deep Linear Probe Generators (ProbeGen), a simple and effective modification to probing approaches. ProbeGen adds a shared generator module with a deep linear architecture, providing an inductive bias towards structured probes thus reducing overfitting. While simple, ProbeGen performs significantly better than the state-of-the-art and is very efficient, requiring between 30 to 1000 times fewer FLOPs than other top approaches.