mlsanityData to 7 Oct 2026

Confidence Dimension for Deep Learning based on Hoeffding Inequality and Relative Evaluation

arXiv 2203.09082 ↗First posted 17 Mar 2022cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Research on the generalization ability of deep neural networks (DNNs) has recently attracted a great deal of attention. However, due to their complex architectures and large numbers of parameters, measuring the generalization ability of specific DNN models remains an open challenge. In this paper, we propose to use multiple factors to measure and rank the relative generalization of DNNs based on a new concept of confidence dimension (CD). Furthermore, we provide a feasible framework in our CD to theoretically calculate the upper bound of generalization based on the conventional Vapnik-Chervonenk dimension (VC-dimension) and Hoeffding's inequality. Experimental results on image classification and object detection demonstrate that our CD can reflect the relative generalization ability for different DNNs. In addition to full-precision DNNs, we also analyze the generalization ability of binary neural networks (BNNs), whose generalization ability remains an unsolved problem. Our CD yields a consistent and reliable measure and ranking for both full-precision DNNs and BNNs on all the tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 60 references that are in the map. First posted on arXiv 17 Mar 2022.

Builds on

32 of 60 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  8. 2017Decoupled Weight Decay Regularizationcited by 1,722
  9. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  11. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  12. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
20 more
  1. 2017Exploring Generalization in Deep Learningcited by 595
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  3. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  4. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  5. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
  6. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
  7. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  8. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  9. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 183
  10. 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 180
  11. 2018Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networkscited by 140
  12. 2018Training and Inference with Integers in Deep Neural Networkscited by 84
  13. 2020In Search of Robust Measures of Generalizationcited by 79
  14. 2017Oriented Response Networkscited by 67
  15. 2019Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimizationcited by 58
  16. 2019On Generalization Error Bounds of Noisy Gradient Methods for Non-Convex Learningcited by 57
  17. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  18. 2020Understanding Why Neural Networks Generalize Well Through GSNR of Parameterscited by 40
  19. 2020Generalization bounds for deep learningcited by 38
  20. 2018Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagationcited by 29

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Reliability and Interpretability in Science and Deep Learningcited by 2