mlsanityData to 7 Oct 2026

Explicit Regularisation in Gaussian Noise Injections

arXiv 2007.07368 ↗First posted 14 Jul 2020stat.ML cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have been few studies on understanding the regularising effect they induce when applied to network activations. Here we derive the explicit regulariser of GNIs, obtained by marginalising out the injected noise, and show that it penalises functions with high-frequency components in the Fourier domain; particularly in layers closer to a neural network's output. We show analytically and empirically that such regularisation produces calibrated classifiers with large classification margins.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 8; 2024: 3; 2025: 8; 2026: 4.

37 papers.

Sort:

By area:

  1. 2022Overview frequency principle/spectral bias in deep learningcited by 68
  2. 2021Noisy Recurrent Neural Networkscited by 42
  3. 2022Word Order Does Matter (And Shuffled Language Models Know It)cited by 32
  4. 2021On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implicationscited by 31
  5. 2021Reliably fast adversarial training via latent adversarial perturbationcited by 29
  6. 2021Noisy Feature Mixupcited by 27
  7. 2022Explicit Regularization in Overparametrized Models via Noise Injectioncited by 25
  8. 2023Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descentcited by 25
  9. 2023FedFA: Federated Feature Augmentationcited by 22
  10. 2022NoisyMix: Boosting Model Robustness to Common Corruptionscited by 17
  11. 2023Exploiting Noise as a Resource for Computation and Learning in Spiking Neural Networkscited by 17
  12. 2023Noise Stability Optimization for Finding Flat Minima: A Hessian-based Regularization Approachcited by 14
25 more
  1. 2021Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injectionscited by 12
  2. 2024GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMscited by 8
  3. 2025Stochastic Preconditioning for Neural Field Optimizationcited by 8
  4. 2021Structured Dropout Variational Inference for Bayesian Neural Networkscited by 7
  5. 2021Regularization in ResNet with Stochastic Depthcited by 7
  6. 2025MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generationcited by 7
  7. 2025Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clientscited by 5
  8. 2023Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networkscited by 4
  9. 2023PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusioncited by 3
  10. 2024Towards Unraveling and Improving Generalization in World Modelscited by 3
  11. 2026Language models recognize dropout and Gaussian noise applied to their activationscited by 3
  12. 2025Deep Neural Networks Inspired by Differential Equationscited by 2
  13. 2022Stochastic Perturbations of Tabular Features for Non-Deterministic Inference with Automungecited by 1
  14. 2023Ghost Noise for Regularizing Deep Neural Networkscited by 1
  15. 2023DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolutioncited by 1
  16. 2024MeMo: Meaningful, Modular Controllers via Noise Injectioncited by 1
  17. 2025Overlap-Adaptive Regularization for Conditional Average Treatment Effect Estimationcited by 1
  18. 2021Variational Autoencoders: A Harmonic Perspectivecited by 0
  19. 2022On gradient descent training under data augmentation with on-line noisy copiescited by 0
  20. 2025Relevance-driven Input Dropout: an Explanation-guided Regularization Techniquecited by 0
  21. 2025DenoGrad: A Gradient-Based Framework for Data Refinement in Tabular and Time-Series Learningcited by 0
  22. 2025Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisationcited by 0
  23. 2026Information Hidden in Gradients of Regression with Target Noisecited by 0
  24. 2026NOBLE: Accelerating Transformers with Nonlinear Low-Rank Branchescited by 0
  25. 2026PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networkscited by 0