mlsanityData to 7 Oct 2026

Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning

arXiv 2603.17365 ↗First posted 18 Mar 2026cs.LG math.PRcited by 0

Not on the map. Why?

Abstract (arXiv)

Internal noise in deep networks is usually inherited from heuristics such as dropout, hard masking, or additive perturbation. We ask two questions: what correlation geometry should internal noise have, and is the implemented perturbation compatible with the representations it acts on? We answer these questions through Variational Kernel Design (VKD), a framework in which a noise mechanism is specified by a law family, a correlation kernel, and an injection operator, and is derived from learning desiderata. In a solved spatial subfamily, a quadratic maximum-entropy principle over latent log-fields yields a Gaussian optimizer with precision given by the Dirichlet Laplacian, so the induced geometry is the Dirichlet Green kernel. Wick normalization then gives a canonical positive mean-one gate, Gaussian Chaos Noise (GCh). For the sample-wise gate used in practice, we prove exact Gaussian control of pairwise log-ratio deformation, margin-sensitive ranking stability, and an exact expected intrinsic roughness budget; hard binary masks instead induce singular or coherence-amplified distortions on positive coherent representations. On ImageNet and ImageNet-C, GCh consistently improves calibration and under shift also improves NLL at competitive accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

24 of 37 references.

References · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  9. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  10. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
12 more
  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2019When Does Label Smoothing Help?cited by 985
  4. 2016Deep Networks with Stochastic Depthcited by 973
  5. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  6. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  7. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  8. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  9. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  10. 2018DropBlock: A regularization method for convolutional networkscited by 261
  11. 2018ShakeDrop Regularization for Deep Residual Learningcited by 40
  12. 2022PatchDropout: Economizing Vision Transformers Using Patch Dropoutcited by 28

Built on by

Not cited yet.