# Efficient estimation of the ANOVA mean dimension, with an application to neural net classification

arXiv: https://arxiv.org/abs/2007.01281 · HTML: https://mlsanity.com/paper/2007.01281/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 2 of 2)

- [The Mean Dimension of Neural Networks -- What causes the interaction effects?](https://mlsanity.com/paper/2207.04890/) (2022)
- [The twin peaks of learning neural networks](https://mlsanity.com/paper/2401.12610/) (2024)

## Abstract (arXiv)

The mean dimension of a black box function of d variables is a convenient way to summarize the extent to which it is dominated by high or low order interactions. It is expressed in terms of 2d-1 variance components but it can be written as the sum of d Sobol' indices that can be estimated by leave one out methods. We compare the variance of these leave one out methods: a Gibbs sampler called winding stairs, a radial sampler that changes each variable one at a time from a baseline, and a naive sampler that never reuses function evaluations and so costs about double the other methods. For an additive function the radial and winding stairs are most efficient. For a multiplicative function the naive method can easily be most efficient if the factors have high kurtosis. As an illustration we consider the mean dimension of a neural network classifier of digits from the MNIST data set. The classifier is a function of 784 pixels. For that problem, winding stairs is the best algorithm. We find that inputs to the final softmax layer have mean dimensions ranging from 1.35 to 2.0.