mlsanityData to 7 Oct 2026

Probabilistic computation and uncertainty quantification with emerging covariance

arXiv 2305.19265 ↗First posted 30 May 2023cs.LG cs.NE math.STcited by 0

Not on the map. Why?

Abstract (arXiv)

Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 51 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

21 of 51 references.

References · 21

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  8. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  9. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  10. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
9 more
  1. 2016Benefits of depth in neural networkscited by 248
  2. 2019DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networkscited by 92
  3. 2018Bayesian Uncertainty Estimation for Batch Normalized Deep Networkscited by 70
  4. 2017Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classificationcited by 56
  5. 2016Unsupervised Risk Estimation Using Only Conditional Independence Structurecited by 17
  6. 2021Bridging the Gap Between Explainable AI and Uncertainty Quantification to Enhance Trustabilitycited by 11
  7. 2021Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noisecited by 2
  8. 2023Toward stochastic neural computingcited by 2
  9. 2015Correlated fluctuations in strongly-coupled binary networks beyond equilibriumcited by 1

Built on by

Not cited yet.