Distribution-restrained Softmax Loss for the Model Robustness
Not on the map. Why?
Abstract (arXiv)
Recently, the robustness of deep learning models has received widespread attention, and various methods for improving model robustness have been proposed, including adversarial training, model architecture modification, design of loss functions, certified defenses, and so on. However, the principle of the robustness to attacks is still not fully understood, also the related research is still not sufficient. Here, we have identified a significant factor that affects the robustness of models: the distribution characteristics of softmax values for non-real label samples. We found that the results after an attack are highly correlated with the distribution characteristics, and thus we proposed a loss function to suppress the distribution diversity of softmax. A large number of experiments have shown that our method can improve robustness without significant time consumption.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 43 references that are in the map. First posted on arXiv 22 Mar 2023.
Builds on
31 of 43 references.
References · 31
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2016Adversarial examples in the physical worldcited by 1,769
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
19 more
- 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
- 2016Adversarial Machine Learning at Scalecited by 936
- 2018Robustness May Be at Odds with Accuracycited by 880
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
- 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
- 2020Overfitting in adversarially robust deep learningcited by 506
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
- 2018Certified Defenses against Adversarial Examplescited by 306
- 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
- 2019AdvHat: Real-world adversarial attack on ArcFace Face ID systemcited by 163
- 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146
- 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
- 2019Adversarial Training Can Hurt Generalizationcited by 132
- 2017Robust Physical-World Attacks on Deep Learning Modelscited by 72
- 2019Fooling a Real Car with Adversarial Traffic Signscited by 29
- 2017Residual Convolutional CTC Networks for Automatic Speech Recognitioncited by 21
- 2018Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attackscited by 13
- 2022Understanding the Impact of Adversarial Robustness on Accuracy Disparitycited by 9
- 2021Perlin Noise Improve Adversarial Robustnesscited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.