mlsanityData to 7 Oct 2026

ShakeDrop Regularization for Deep Residual Learning

arXiv 1802.02375 ↗First posted 7 Feb 2018cs.CVcited by 40

Not on the map. Why?

Abstract (arXiv)

Overfitting is a crucial problem in deep neural networks, even in the latest network architectures. In this paper, to relieve the overfitting effect of ResNet and its improvements (i.e., Wide ResNet, PyramidNet, and ResNeXt), we propose a new regularization method called ShakeDrop regularization. ShakeDrop is inspired by Shake-Shake, which is an effective regularization method, but can be applied to ResNeXt only. ShakeDrop is more effective than Shake-Shake and can be applied not only to ResNeXt but also ResNet, Wide ResNet, and PyramidNet. An important key is to achieve stability of training. Because effective regularization often causes unstable training, we introduce a training stabilizer, which is an unusual use of an existing regularizer. Through experiments under various conditions, we demonstrate the conditions under which ShakeDrop works well.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 11; 2023: 4; 2024: 6; 2025: 4; 2026: 2.

40 papers.

Sort:

By area:

  1. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  2. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  3. 2022Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networkscited by 55
  4. 2022TeachAugment: Data Augmentation Optimization Using Teacher Knowledgecited by 37
  5. 2021Spectral Bias in Practice: The Role of Function Frequency in Generalizationcited by 36
  6. 2022Bootstrap Generalization Ability from Loss Landscape Perspectivecited by 16
  7. 2021DivAug: Plug-in Automated Data Augmentation with Explicit Diversity Maximizationcited by 15
  8. 2021Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixingcited by 15
  9. 2021Omnizart: A General Toolbox for Automatic Music Transcriptioncited by 13
  10. 2022Improving Robustness with Adaptive Weight Decaycited by 13
  11. 2023When to Learn What: Model-Adaptive Data Augmentation Curriculumcited by 13
  12. 2021MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurationscited by 12
28 more
  1. 2021CAM-loss: Towards Learning Spatially Discriminative Feature Representationscited by 12
  2. 2022A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Taskscited by 12
  3. 2021Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attackscited by 11
  4. 2024Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Surveycited by 10
  5. 2022Privacy-Preserving Image Classification Using Vision Transformercited by 9
  6. 2022Exploring Temporally Dynamic Data Augmentation for Video Recognitioncited by 9
  7. 2021Improving Model Generalization by Agreement of Learned Representations from Data Augmentationcited by 5
  8. 2024AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Modelscited by 5
  9. 2021DAAS: Differentiable Architecture and Augmentation Policy Searchcited by 4
  10. 2022A Survey on Dropout Methods and Experimental Verification in Recommendationcited by 4
  11. 2021FocusedDropout for Convolutional Neural Networkcited by 3
  12. 2023Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networkscited by 2
  13. 2025Deep Neural Networks Inspired by Differential Equationscited by 2
  14. 2024ContextMix: A context-aware data augmentation method for industrial visual inspection systemscited by 1
  15. 2024MaskBlur: Spatial and Angular Data Augmentation for Light Field Image Super-Resolutioncited by 1
  16. 2025Boosting Ray Search Procedure of Hard-label Attacks with Transfer-based Priorscited by 1
  17. 2025Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Maskingcited by 1
  18. 2025IPF-RDA: An Information-Preserving Framework for Robust Data Augmentationcited by 1
  19. 2021Selective Output Smoothing Regularization: Regularize Neural Networks by Softening Output Distributionscited by 0
  20. 2021Optimizing Data Augmentation Policy Through Random Unidimensional Searchcited by 0
  21. 2022A Pixel-based Encryption Method for Privacy-Preserving Deep Learning Modelscited by 0
  22. 2022Macro-block dropout for improved regularization in training end-to-end speech recognition modelscited by 0
  23. 2023Unproportional mosaicingcited by 0
  24. 2023Improving Generalization in Meta-Learning via Meta-Gradient Augmentationcited by 0
  25. 2024Efficient Higher-order Convolution for Small Kernels in Deep Learningcited by 0
  26. 2024Optimal Layer Selection for Latent Data Augmentationcited by 0
  27. 2026Gradient Residual Connectionscited by 0
  28. 2026Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learningcited by 0