mlsanityData to 7 Oct 2026

Enhancing Classifier Conservativeness and Robustness by Polynomiality

arXiv 2203.12693 ↗First posted 23 Mar 2022cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We illustrate the detrimental effect, such as overconfident decisions, that exponential behavior can have in methods like classical LDA and logistic regression. We then show how polynomiality can remedy the situation. This, among others, leads purposefully to random-level performance in the tails, away from the bulk of the training data. A directly related, simple, yet important technical novelty we subsequently present is softRmax: a reasoned alternative to the standard softmax function employed in contemporary (deep) neural networks. It is derived through linking the standard softmax to Gaussian class-conditional models, as employed in LDA, and replacing those by a polynomial alternative. We show that two aspects of softRmax, conservativeness and inherent gradient regularization, lead to robustness against adversarial attacks without gradient obfuscation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 23 Mar 2022.

Builds on

25 of 43 references.

References · 25

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Adversarial examples in the physical worldcited by 1,762
  4. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  5. 2016Adversarial Machine Learning at Scalecited by 934
  6. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  7. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  8. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  9. 2013Do Deep Nets Really Need to be Deep?cited by 466
  10. 2019On Evaluating Adversarial Robustnesscited by 442
  11. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  12. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
13 more
  1. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  2. 2019A review of domain adaptation without target labelscited by 154
  3. 2016Towards the Science of Security and Privacy in Machine Learningcited by 89
  4. 2019Scaleable input gradient regularization for adversarial robustnesscited by 45
  5. 2018Rethinking Feature Distribution for Loss Functions in Image Classificationcited by 39
  6. 2015An Exploration of Softmax Alternatives Belonging to the Spherical Loss Familycited by 27
  7. 2017Biologically inspired protection of deep networks from adversarial attackscited by 23
  8. 2018Sigsoftmax: Reanalysis of the Softmax Bottleneckcited by 19
  9. 2018Max-Mahalanobis Linear Discriminant Analysis Networkscited by 18
  10. 2016One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilitiescited by 14
  11. 2020Softmax-based Classification is k-means Clustering: Formal Proof, Consequences for Adversarial Attacks, and Improvement through Centroid Based Tailoringcited by 8
  12. 2021Impact of Attention on Adversarial Robustness of Image Classification Modelscited by 4
  13. 2013Saturating Auto-Encoderscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Fairness Increases Adversarial Vulnerabilitycited by 5