mlsanityData to 7 Oct 2026

Effective dimension of machine learning models

arXiv 2112.04807 ↗First posted 9 Dec 2021cs.LG stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

Making statements about the performance of trained models on tasks involving new data is one of the primary goals of machine learning, i.e., to understand the generalization power of a model. Various capacity measures try to capture this ability, but usually fall short in explaining important characteristics of models that we observe in practice. In this study, we propose the local effective dimension as a capacity measure which seems to correlate well with generalization error on standard data sets. Importantly, we prove that the local effective dimension bounds the generalization error and discuss the aptness of this capacity measure for machine learning models.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

24 of 46 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  5. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  6. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  7. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  8. 2017Exploring Generalization in Deep Learningcited by 595
  9. 2019Importance Estimation for Neural Network Pruningcited by 531
  10. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  11. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  12. 2020The power of quantum neural networkscited by 377
12 more
  1. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  2. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  3. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
  4. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 213
  5. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 213
  6. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  7. 2017Fisher-Rao Metric, Geometry, and Complexity of Neural Networkscited by 99
  8. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  9. 2020A scale-dependent notion of effective dimensioncited by 17
  10. 2018Identifying Generalization Properties in Neural Networkscited by 15
  11. 2018Approximate Fisher Information Matrix to Characterise the Training of Deep Neural Networkscited by 12
  12. 2018Compact and Computationally Efficient Representation of Deep Neural Networkscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 3; 2025: 4; 2026: 2.

14 papers.

Sort:

  1. 2023Deep networks for system identification: a Surveycited by 35
  2. 2022Do Quantum Circuit Born Machines Generalize?cited by 19
  3. 2023Generalization of Quantum Machine Learning Models Using Quantum Fisher Information Metriccited by 10
  4. 2024Understanding Generalization in Quantum Machine Learning with Marginscited by 6
  5. 2022Quantum neural networks force fields generationcited by 4
  6. 2023Quantum Machine Learning for Malware Classificationcited by 3
  7. 2024A Two-Scale Complexity Measure for Deep Learning Modelscited by 3
  8. 2025Generalization Bounds in Hybrid Quantum-Classical Machine Learning Modelscited by 3
  9. 2025A universal compression theory for lottery ticket hypothesis and neural scaling lawscited by 2
  10. 2024Data-Dependent Generalization Bounds for Parameterized Quantum Models Under Noisecited by 1
  11. 2025Quantum feature-map learning with reduced resource overheadcited by 1
  12. 2025Fisher Information, Training and Bias in Fourier Regression Modelscited by 0
2 more
  1. 2026Epistemic Compression: The Case for Deliberate Ignorance in High-Stakes AIcited by 0
  2. 2026Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspacescited by 0