Wide Neural Networks as a Baseline for the Computational No-Coincidence Conjecture
Not on the map. Why?
Abstract (arXiv)
We establish that randomly initialized neural networks, with large width and a natural choice of hyperparameters, have nearly independent outputs exactly when their activation function is nonlinear with zero mean under the Gaussian measure: mathbbEz ~ N(0,1)[σ(z)]=0. For example, this includes ReLU and GeLU with an additive shift, as well as tanh, but not ReLU or GeLU by themselves. Because of their nearly independent outputs, we propose neural networks with zero-mean activation functions as a promising candidate for the Alignment Research Center's computational no-coincidence conjecture -- a conjecture that aims to measure the limits of AI interpretability.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 8 Oct 2025.
Builds on
12 of 19 references.
References · 12
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2017Deep Neural Networks as Gaussian Processescited by 622
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
- 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
- 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
- 2025Detecting Strategic Deception Using Linear Probescited by 68
- 2022Planting Undetectable Backdoors in Machine Learning Modelscited by 59
- 2024Neural Redshift: Random Networks are not Random Functionscited by 43
- 2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 34
- 2024Backdoor defense, learnability and obfuscationcited by 2
- 2025Pseudorandomness Properties of Random Reversible Circuitscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.