# Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables

arXiv: https://arxiv.org/abs/1703.00091 · HTML: https://mlsanity.com/paper/1703.00091/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2017.

## Built on by (most cited first, 20 of 20)

- [LogME: Practical Assessment of Pre-trained Models for Transfer Learning](https://mlsanity.com/paper/2102.11005/) (2021)
- [Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse](https://mlsanity.com/paper/2206.03126/) (2022)
- [Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discrimination](https://mlsanity.com/paper/2206.01535/) (2022)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs](https://mlsanity.com/paper/2110.10545/) (2021)
- [Low-rank extended Kalman filtering for online learning of neural networks from streaming data](https://mlsanity.com/paper/2305.19535/) (2023)
- [An Analytic Solution to Covariance Propagation in Neural Networks](https://mlsanity.com/paper/2403.16163/) (2024)
- [Massively Scaling Heteroscedastic Classifiers](https://mlsanity.com/paper/2301.12860/) (2023)
- [Enhancing Privacy against Inversion Attacks in Federated Learning by using Mixing Gradients Strategies](https://mlsanity.com/paper/2204.12495/) (2022)
- [Rethinking Approximate Gaussian Inference in Classification](https://mlsanity.com/paper/2502.03366/) (2025)
- [Uncertainty Propagation in Convolutional Neural Networks: Technical Report](https://mlsanity.com/paper/2102.06064/) (2021)
- [Learning Functional Distributional Semantics with Visual Data](https://mlsanity.com/paper/2204.10624/) (2022)
- [Propagating Variational Model Uncertainty for Bioacoustic Call Label Smoothing](https://mlsanity.com/paper/2210.10526/) (2022)
- [Towards Automated Animal Density Estimation with Acoustic Spatial Capture-Recapture](https://mlsanity.com/paper/2308.12859/) (2023)
- [Distributed Variational Inference for Online Supervised Learning](https://mlsanity.com/paper/2309.02606/) (2023)
- [Single-shot Bayesian approximation for neural networks](https://mlsanity.com/paper/2308.12785/) (2023)
- [Training of Spiking Neural Networks with Expectation-Propagation](https://mlsanity.com/paper/2506.23757/) (2025)
- [α-TCAV: A Unified Framework for Testing with Concept Activation Vectors](https://mlsanity.com/paper/2605.15688/) (2026)
- [Calibrated Sampling-Free Uncertainty Estimation in Bayesian Deep Learning](https://mlsanity.com/paper/2606.16214/) (2026)

## Abstract (arXiv)

This note is concerned with accurate and computationally efficient approximations of moments of Gaussian random variables passed through sigmoid or softmax mappings. These approximations are semi-analytical (i.e. they involve the numerical adjustment of parametric forms) and highly accurate (they yield 5% error at most). We also highlight a few niche applications of these approximations, which arise in the context of, e.g., drift-diffusion models of decision making or non-parametric data clustering approaches. We provide these as examples of efficient alternatives to more tedious derivations that would be needed if one was to approach the underlying mathematical issues in a more formal way. We hope that this technical note will be helpful to modellers facing similar mathematical issues, although maybe stemming from different academic prospects.