mlsanityData to 7 Oct 2026

Towards Task and Architecture-Independent Generalization Gap Predictors

arXiv 1906.01550 ↗First posted 4 Jun 2019stat.ML cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Can we use deep learning to predict when deep learning works? Our results suggest the affirmative. We created a dataset by training 13,500 neural networks with different architectures, on different variations of spiral datasets, and using different optimization parameters. We used this dataset to train task-independent and architecture-independent generalization gap predictors for those neural networks. We extend Jiang et al. (2018) to also use DNNs and RNNs and show that they outperform the linear model, obtaining R2=0.965. We also show results for architecture-independent, task-independent, and out-of-distribution generalization gap prediction tasks. Both DNNs and RNNs consistently and significantly outperform linear models, with RNNs obtaining R2=0.584.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 3; 2023: 5; 2024: 3; 2025: 2.

18 papers.

Sort:

  1. 2021Assessing Generalization of SGD via Disagreementcited by 116
  2. 2023Permutation Equivariant Neural Functionalscited by 69
  3. 2023Neural Functional Transformerscited by 50
  4. 2024Towards Scalable and Versatile Weight Space Learningcited by 40
  5. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 38
  6. 2022Multi-Object Navigation with dynamically learned neural implicit representationscited by 21
  7. 2023On the Importance of Feature Separability in Predicting Out-Of-Distribution Errorcited by 15
  8. 2021Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Predictioncited by 9
  9. 2024Improving Hyperparameter Optimization with Checkpointed Model Weightscited by 5
  10. 2023Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Modelscited by 4
  11. 2024Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shiftcited by 3
  12. 2025A Model Zoo on Phase Transitions in Neural Networkscited by 3
6 more
  1. 2021In Search of Probeable Generalization Measurescited by 2
  2. 2022Exploiting Explainable Metrics for Augmented SGDcited by 2
  3. 2021Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examplescited by 1
  4. 2021Spectral Roll-off Points Variations: Exploring Useful Information in Feature Maps by Its Variationscited by 0
  5. 2023Can we infer the presence of Differential Privacy in Deep Learning models' weights? Towards more secure Deep Learningcited by 0
  6. 2025Can We Optimize Deep RL Policy Weights as Trajectory Modeling?cited by 0