mlsanityData to 14 Oct 2026

Understanding Deep Learning via Decision Boundary

arXiv 2206.01515 ↗First posted 3 Jun 2022cs.LG cs.AI stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper discovers that the neural network with lower decision boundary (DB) variability has better generalizability. Two new notions, algorithm DB variability and (ε, η)-data DB variability, are proposed to measure the decision boundary variability from the algorithm and data perspectives. Extensive experiments show significant negative correlations between the decision boundary variability and the generalizability. From the theoretical view, two lower bounds based on algorithm DB variability are proposed and do not explicitly depend on the sample size. We also prove an upper bound of order Oleft(frac1√m+ε+ηlogfrac1ηright) based on data DB variability. The bound is convenient to estimate without the requirement of labels, and does not explicitly depend on the network size which is usually prohibitively large in deep learning.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 52 of its 75 references that are in the map. First posted on arXiv 3 Jun 2022.

Builds on

52 of 75 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  8. 2013Intriguing properties of neural networkscited by 5,155
  9. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  10. 2016Wide Residual Networkscited by 3,070
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
38 more
  1. 2018On the Spectral Bias of Neural Networkscited by 1,341
  2. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  3. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  4. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  5. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  6. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 552
  7. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  8. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  9. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 363
  10. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
  11. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
  12. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 320
  13. 2017How to Escape Saddle Points Efficientlycited by 285
  14. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  15. 2016Deep Learning without Poor Local Minimacited by 234
  16. 2019SGD on Neural Networks Learns Functions of Increasing Complexitycited by 204
  17. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 198
  18. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  19. 2020Directional convergence and alignment in deep learningcited by 178
  20. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  21. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  22. 2021Assessing Generalization of SGD via Disagreementcited by 117
  23. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 106
  24. 2014Collaborative Deep Learning for Recommender Systemscited by 95
  25. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 95
  26. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 57
  27. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  28. 2020Recent advances in deep learning theorycited by 41
  29. 2019The Local Elasticity of Neural Networkscited by 37
  30. 2019Adversarial Attack on Skeleton-based Human Action Recognitioncited by 32
  31. 2019Characterizing the Decision Boundary of Deep Neural Networkscited by 32
  32. 2020Hold me tight! Influence of discriminative features on deep network boundariescited by 31
  33. 2017Depth Creates No Bad Local Minimacited by 29
  34. 2019Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theorycited by 21
  35. 2020Understanding the Decision Boundary of Deep Neural Networks: An Empirical Studycited by 20
  36. 2020Piecewise linear activations substantially shape the loss surfaces of neural networkscited by 19
  37. 2020Benign Overfitting and Noisy Featurescited by 18
  38. 2020Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundarycited by 13

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 0; 2025: 2; 2026: 1.

7 papers.

  1. 2023Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classificationcited by 11
  2. 2023Image Captions are Natural Prompts for Text-to-Image Modelscited by 1
  3. 2023Improving classifier decision boundaries using nearest neighborscited by 1
  4. 2025The Easy Path to Robustness: Coreset Selection using Sample Hardnesscited by 1
  5. 2022The Vanishing Decision Boundary Complexity and the Strong First Componentcited by 0
  6. 2025Diffusion-based Counterfactual Augmentation: Towards Robust and Interpretable Knee Osteoarthritis Gradingcited by 0
  7. 2026GJDNet: Robust Graph Neural Networks via Joint Disentangled Learning Against Adversarial Attackscited by 0