mlsanityData to 7 Oct 2026

Predicting Parameters in Deep Learning

arXiv 1306.0543 ↗First posted 3 Jun 2013cs.LG cs.NE stat.MLcited by 223

Not on the map. Why?

Abstract (arXiv)

We demonstrate that there is significant redundancy in the parameterization of several deep learning models. Given only a few weight values for each feature it is possible to accurately predict the remaining values. Moreover, we show that not only can the parameter values be predicted, but many of them need not be learned at all. We train several different architectures by learning only a small number of weights and predicting the rest. In the best case we are able to predict more than 95% of the weights of a network without any drop in accuracy.

Key facts. Cited by 223 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jun 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 60; 2022: 48; 2023: 42; 2024: 30; 2025: 28; 2026: 14.

223 papers.

Sort:

By area:

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Dynamic Neural Networks: A Surveycited by 405
  3. 2024Task Singular Vectors: Reducing Task Interference in Model Mergingcited by 143
  4. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  5. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  6. 2024Convolutional Kolmogorov-Arnold Networkscited by 76
  7. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 70
  8. 2022Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weightscited by 63
  9. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  10. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  11. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  12. 2022Blueprint Separable Residual Network for Efficient Image Super-Resolutioncited by 44
38 more
  1. 2023Ankh: Optimized Protein Language Model Unlocks General-Purpose Modellingcited by 44
  2. 2024A Survey on Transformer Compressioncited by 44
  3. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  4. 2024Neural Network Diffusioncited by 39
  5. 2024OLoRA: Orthonormal Low-Rank Adaptation of Large Language Modelscited by 39
  6. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 38
  7. 2024Diffusion-Based Neural Network Weights Generationcited by 38
  8. 2022On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Modelscited by 34
  9. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
  10. 2023Directional Connectivity-based Segmentation of Medical Imagescited by 30
  11. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  12. 2021Deep Neural Networks as Point Estimates for Deep Gaussian Processescited by 26
  13. 2023ResFields: Residual Neural Fields for Spatiotemporal Signalscited by 26
  14. 2024Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptationcited by 26
  15. 2023Q-DETR: An Efficient Low-Bit Quantized Detection Transformercited by 23
  16. 2022Data-Efficient Structured Pruning via Submodular Optimizationcited by 22
  17. 2021ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximationscited by 21
  18. 2024Spectral Adapter: Fine-Tuning in Spectral Spacecited by 21
  19. 2021Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolutioncited by 20
  20. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  21. 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 19
  22. 2023A Rainbow in Deep Network Black Boxescited by 18
  23. 2021Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularizationcited by 17
  24. 2022Defending against the Label-flipping Attack in Federated Learningcited by 16
  25. 2023Backdoor Federated Learning by Poisoning Backdoor-Critical Layerscited by 16
  26. 2021Truly Sparse Neural Networks at Scalecited by 15
  27. 2021ECQx: Explainability-Driven Quantization for Low-Bit and Sparse DNNscited by 15
  28. 2023MetaGait: Learning to Learn an Omni Sample Adaptive Representation for Gait Recognitioncited by 15
  29. 2021Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generationcited by 14
  30. 2021Convolutional Neural Network Compression through Generalized Kronecker Product Decompositioncited by 14
  31. 2021Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networkscited by 14
  32. 2022Structural Knowledge Distillation for Object Detectioncited by 14
  33. 2023Tensor Decomposition for Model Reduction in Neural Networks: A Reviewcited by 14
  34. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  35. 2021InsPose: Instance-Aware Networks for Single-Stage Multi-Person Pose Estimationcited by 13
  36. 2021Neural Network Quantization for Efficient Inference: A Surveycited by 13
  37. 2022Learning Compressed Embeddings for On-Device Inferencecited by 13
  38. 2023Bi-ViT: Pushing the Limit of Vision Transformer Quantizationcited by 13

50 most cited of 223 All 223 (JSON)