mlsanityData to 14 Oct 2026

Deep Linear Probe Generators for Weight Space Learning

arXiv 2410.10811 ↗First posted 14 Oct 2024cs.LG cs.CVcited by 17

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Weight space learning aims to extract information about a neural network, such as its training dataset or generalization error. Recent approaches learn directly from model weights, but this presents many challenges as weights are high-dimensional and include permutation symmetries between neurons. An alternative approach, Probing, represents a model by passing a set of learned inputs (probes) through the model, and training a predictor on top of the corresponding outputs. Although probing is typically not used as a stand alone approach, our preliminary experiment found that a vanilla probing baseline worked surprisingly well. However, we discover that current probe learning strategies are ineffective. We therefore propose Deep Linear Probe Generators (ProbeGen), a simple and effective modification to probing approaches. ProbeGen adds a shared generator module with a deep linear architecture, providing an inductive bias towards structured probes thus reducing overfitting. While simple, ProbeGen performs significantly better than the state-of-the-art and is very efficient, requiring between 30 to 1000 times fewer FLOPs than other top approaches.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 50 of its 62 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

50 of 62 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  10. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  11. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  12. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
38 more
  1. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  2. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  3. 2016HyperNetworkscited by 1,027
  4. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  5. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  6. 2019Implicit Regularization in Deep Matrix Factorizationcited by 405
  7. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  8. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  9. 2024Stealing Part of a Production Language Modelcited by 159
  10. 2017Optimizing the Latent Space of Generative Networkscited by 157
  11. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  12. 2020Predicting Neural Network Accuracy from Weightscited by 110
  13. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
  14. 2021Learning to See by Looking at Noisecited by 92
  15. 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 86
  16. 2023Permutation Equivariant Neural Functionalscited by 70
  17. 2020Classifying the classifier: dissecting the weight space of neural networkscited by 65
  18. 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
  19. 2022Re-basin via implicit Sinkhorn differentiationcited by 62
  20. 2024Graph Neural Networks for Learning Equivariant Representations of Neural Networkscited by 59
  21. 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 57
  22. 2023Neural Functional Transformerscited by 52
  23. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  24. 2024Towards Scalable and Versatile Weight Space Learningcited by 41
  25. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 39
  26. 2024Interpreting the Weight Space of Customized Diffusion Modelscited by 36
  27. 2023Equivariant Deep Weight Space Alignmentcited by 35
  28. 2024Improved Generalization of Weight Space Networks via Augmentationscited by 24
  29. 2024Scale Equivariant Graph Metanetworkscited by 24
  30. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 22
  31. 2024Equivariant Neural Functional Networks for Transformerscited by 22
  32. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  33. 2024Recovering the Pre-Fine-Tuning Weights of Generative Modelscited by 16
  34. 2024Learning Useful Representations of Recurrent Neural Network Weight Matricescited by 15
  35. 2022NeRN -- Learning Neural Representations for Neural Networkscited by 14
  36. 2023How to Train Neural Field Representations: A Comprehensive Study and Benchmarkcited by 12
  37. 2024Unsupervised Model Tree Heritage Recoverycited by 11
  38. 2024Dataset Size Recovery from LoRA Weightscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 10.

17 papers.

Sort:

By area:

  1. 2025We Should Chart an Atlas of All the World's Modelscited by 25
  2. 2024Learning on Model Weights using Tree Expertscited by 7
  3. 2025GradMetaNet: An Equivariant Architecture for Learning on Gradientscited by 7
  4. 2025Weight-Space Linear Recurrent Neural Networkscited by 5
  5. 2025Learning Model Representations Using Publicly Available Model Hubscited by 4
  6. 2025A Model Zoo of Vision Transformerscited by 3
  7. 2026What Linear Probes Miss: Multi-View Probing for Weight-Space Learningcited by 2
  8. 2026Discovering Hidden Gems in Model Repositoriescited by 1
  9. 2026Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theoremcited by 1
  10. 2026Diagnosing Knowledge Conflict in Multimodal Long-Chain Reasoningcited by 1
  11. 2026Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAMcited by 1
  12. 2025Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliancecited by 0
5 more
  1. 2026ProjLens: Unveiling the Role of Projectors in Multimodal Model Safetycited by 0
  2. 2026Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?cited by 0
  3. 2026Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiabilitycited by 0
  4. 2026Weight-Space Mixture-of-Experts for Implicit Neural Representation Classificationcited by 0
  5. 2026Finite Probes Suffice: Identifiability and Universality for Weight-Space Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Search Spaces for Neural Model Training
  2. 2023Data Augmentations in Deep Weight Spaces
  3. 2026Deep Minds and Shallow Probes
  4. 2026DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weights
  5. 2025Geometric Flow Models over Neural Network Weights
  6. 2026RAPTOR: Ridge-Adaptive Logistic Probes
  7. 2024Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling
  8. 2026APEX: Probing Neural Networks via Activation Perturbation