mlsanityData to 14 Oct 2026

Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization

arXiv 1710.05179 ↗First posted 14 Oct 2017cs.LG cs.CVcited by 56

Not on the map. Why?

Abstract (arXiv)

Overfitting is one of the most critical challenges in deep neural networks, and there are various types of regularization methods to improve generalization performance. Injecting noises to hidden units during training, e.g., dropout, is known as a successful regularizer, but it is still not clear enough why such training techniques work well in practice and how we can maximize their benefit in the presence of two conflicting objectives---optimizing to true data distribution and preventing overfitting by regularization. This paper addresses the above issues by 1) interpreting that the conventional training methods with regularization by noise injection optimize the lower bound of the true objective and 2) proposing a technique to achieve a tighter lower bound using multiple noise samples per training example in a stochastic gradient descent iteration. We demonstrate the effectiveness of our idea in several computer vision applications.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 12; 2023: 12; 2024: 12; 2025: 10; 2026: 6.

56 papers.

Sort:

By area:

  1. 2022Quantum Vision Transformerscited by 48
  2. 2023Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based acceleratorscited by 43
  3. 2021Noisy Recurrent Neural Networkscited by 42
  4. 2023A Path to Simpler Models Starts With Noisecited by 33
  5. 2022Cross-View Cross-Scene Multi-View Crowd Countingcited by 28
  6. 2023On The Impact of Machine Learning Randomness on Group Fairnesscited by 26
  7. 2023Exploiting Noise as a Resource for Computation and Learning in Spiking Neural Networkscited by 18
  8. 2023Universal Multimodal Representation for Language Understandingcited by 14
  9. 2022Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Networkcited by 10
  10. 2022A Kernel-Expanded Stochastic Neural Networkcited by 9
  11. 2022ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inferencecited by 9
  12. 2021AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Acceleratorcited by 7
38 more
  1. 2023The stochastic digital human is now enrolling for in silico imaging trials -- Methods and tools for generating digital cohortscited by 6
  2. 2023Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamicscited by 6
  3. 2023Are Ensembles Getting Better all the Time?cited by 6
  4. 2024Phase-driven Domain Generalizable Learning for Nonstationary Time Seriescited by 5
  5. 2024SoK: A Review of Differentially Private Linear Models For High-Dimensional Datacited by 5
  6. 2022Supervision Adaptation Balancing In-distribution Generalization and Out-of-distribution Detectioncited by 4
  7. 2023Training neural networks with structured noise improves classification and generalizationcited by 4
  8. 2023Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networkscited by 4
  9. 2024Syn2Real Domain Generalization for Underwater Mine-like Object Detection Using Side-Scan Sonarcited by 4
  10. 2025Gradient Extrapolation for Debiased Representation Learningcited by 4
  11. 2025Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Modelscited by 4
  12. 2026Spectral methods: crucial for machine learning, natural for quantum computers?cited by 4
  13. 2024Dynamic Cross Attention for Audio-Visual Person Verificationcited by 3
  14. 2024Cross-Attention is Not Always Needed: Dynamic Cross-Attention for Audio-Visual Dimensional Emotion Recognitioncited by 3
  15. 2024A Unified Deep Semantic Expansion Framework for Domain-Generalized Person Re-identificationcited by 3
  16. 2024Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in MLcited by 3
  17. 2021Towards a Robust Differentiable Architecture Search under Label Noisecited by 2
  18. 2022Effect of Batch Normalization on Noise Resistant Property of Deep Learning Modelscited by 2
  19. 2022AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networkscited by 2
  20. 2024SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networkscited by 2
  21. 2024Enhancing the Trainability of Variational Quantum Circuits with Regularization Strategiescited by 2
  22. 2024Greener GRASS: Enhancing GNNs with Encoding, Rewiring, and Attentioncited by 2
  23. 2025Cross-regularization: Adaptive Model Complexity through Validation Gradientscited by 2
  24. 2025Deep Neural Networks Inspired by Differential Equationscited by 2
  25. 2024Artificial Human Intelligence: The role of Humans in the Development of Next Generation AIcited by 1
  26. 2021Analyzing the effectiveness of image augmentations for face recognition from limited datacited by 0
  27. 2022Application of Machine Learning Methods in Inferring Surface Water Groundwater Exchanges using High Temporal Resolution Temperature Measurementscited by 0
  28. 2022Convergence of Batch Updating Methods with Approximate Gradients and/or Noisy Measurements: Theory and Computational Resultscited by 0
  29. 2022Circling Back to Recurrent Models of Languagecited by 0
  30. 2022NEON: Enabling Efficient Support for Nonlinear Operations in Resistive RAM-based Neural Network Acceleratorscited by 0
  31. 2023Context Normalization Layer with Applicationscited by 0
  32. 2023The Benefit of Noise-Injection for Dynamic Gray-Box Model Creationcited by 0
  33. 2024ART: Artifact Removal Transformer for Reconstructing Noise-Free Multichannel Electroencephalographic Signalscited by 0
  34. 2025Using machine learning to map simulated noisy and laser-limited multidimensional spectra to molecular electronic couplingscited by 0
  35. 2025Relevance-driven Input Dropout: an Explanation-guided Regularization Techniquecited by 0
  36. 2025Representation learning with a transformer by contrastive learning for money laundering detectioncited by 0
  37. 2025A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networkscited by 0
  38. 2025VISP: Volatility Informed Stochastic Projection for Adaptive Regularizationcited by 0

50 most cited of 56 All 56 (JSON)