mlsanityData to 14 Oct 2026

The distance between the weights of the neural network is meaningful

arXiv 2102.00396 ↗First posted 31 Jan 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In the application of neural networks, we need to select a suitable model based on the problem complexity and the dataset scale. To analyze the network's capacity, quantifying the information learned by the network is necessary. This paper proves that the distance between the neural network weights in different training stages can be used to estimate the information accumulated by the network in the training process directly. The experiment results verify the utility of this method. An application of this method related to the label corruption is shown at the end.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 46 references that are in the map. First posted on arXiv 31 Jan 2021.

Builds on

18 of 46 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2017Mask R-CNNcited by 7,277
  5. 2014Going Deeper with Convolutionscited by 4,241
  6. 2018YOLOv3: An Incremental Improvementcited by 2,659
  7. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  9. 2016Deep Variational Information Bottleneckcited by 930
  10. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  11. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  12. 2017Wasserstein GANcited by 639
6 more
  1. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  2. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 463
  3. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 214
  4. 2016EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesiscited by 178
  5. 2019Where is the Information in a Deep Neural Network?cited by 42
  6. 2018Understanding Convolutional Neural Networks with Information Theory: An Initial Explorationcited by 32

Built on by

Not cited yet.