mlsanityData to 7 Oct 2026

Deep Convolutional Networks as shallow Gaussian Processes

arXiv 1808.05587 ↗First posted 16 Aug 2018stat.ML cs.LGcited by 137

Not on the map. Why?

Abstract (arXiv)

We show that the output of a (residual) convolutional neural network (CNN) with an appropriate prior over the weights and biases is a Gaussian process (GP) in the limit of infinitely many convolutional filters, extending similar results for dense networks. For a CNN, the equivalent kernel can be computed exactly and, unlike "deep kernels", has very few parameters: only the hyperparameters of the original CNN. Further, we show that this kernel has two properties that allow it to be computed efficiently; the cost of evaluating the kernel for a pair of images is similar to a single forward pass through the original CNN with only one filter per layer. The kernel equivalent to a 32-layer ResNet obtains 0.84% classification error on MNIST, a new record for GPs with a comparable number of parameters.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 30; 2023: 30; 2024: 20; 2025: 8; 2026: 8.

137 papers.

Sort:

By area:

  1. 2021Neural Operator: Learning Maps Between Function Spacescited by 413
  2. 2021Explaining Neural Scaling Lawscited by 353
  3. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  4. 2021Bayesian Neural Network Priors Revisitedcited by 121
  5. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  6. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  7. 2021Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeedcited by 67
  8. 2022Fast Finite Width Neural Tangent Kernelcited by 62
  9. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  10. 2021Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shapingcited by 51
  11. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 43
  12. 2024Neural Redshift: Random Networks are not Random Functionscited by 43
38 more
  1. 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
  2. 2021Scaling Neural Tangent Kernels via Sketching and Random Featurescited by 32
  3. 2021Approximation and Learning with Deep Convolutional Models: a Kernel Perspectivecited by 31
  4. 2021Bayesian Uncertainty Estimation of Learned Variational MRI Reconstructioncited by 30
  5. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 28
  6. 2021Batch Normalization Orthogonalizes Representations in Deep Random Networkscited by 28
  7. 2021Precise characterization of the prior predictive distribution of deep ReLU networkscited by 28
  8. 2021Deep Neural Networks as Point Estimates for Deep Gaussian Processescited by 26
  9. 2022NeuralEF: Deconstructing Kernels by Deep Neural Networkscited by 26
  10. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  11. 2023Neural signature kernels as infinite-width-depth-limits of controlled ResNetscited by 25
  12. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  13. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 24
  14. 2024Super Consistency of Neural Network Landscapes and Learning Rate Transfercited by 23
  15. 2021The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecited by 21
  16. 2023Quantitative CLTs in Deep Neural Networkscited by 19
  17. 2023Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networkscited by 19
  18. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 18
  19. 2022Fast Neural Kernel Embeddings for General Activationscited by 18
  20. 2021Quantum field-theoretic machine learningcited by 16
  21. 2021A theory of representation learning gives a deep generalisation of kernel methodscited by 15
  22. 2021Large-Scale Learning with Fourier Features and Tensor Decompositionscited by 14
  23. 2021Learning curves for Gaussian process regression with power-law priors and targetscited by 14
  24. 2022Evolution of Neural Tangent Kernels under Benign and Adversarial Trainingcited by 14
  25. 2023Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Modelscited by 14
  26. 2021Why flatness does and does not correlate with generalization for deep neural networkscited by 13
  27. 2024Transformers Learn Low Sensitivity Functions: Investigations and Implicationscited by 13
  28. 2022Decomposing neural networks as mappings of correlation functionscited by 12
  29. 2022On the Robustness of Bayesian Neural Networks to Adversarial Attackscited by 12
  30. 2022Globally Gated Deep Linear Networkscited by 12
  31. 2024Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layerscited by 12
  32. 2021Implicit Bias of Linear RNNscited by 11
  33. 2022Learning from Randomly Initialized Neural Network Featurescited by 11
  34. 2022A duality connecting neural network and cosmological dynamicscited by 10
  35. 2022Uncertainty Estimation for Computed Tomography with a Linearised Deep Image Priorcited by 10
  36. 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 10
  37. 2023The Hessian perspective into the Nature of Convolutional Neural Networkscited by 10
  38. 2025From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learningcited by 10

50 most cited of 137 All 137 (JSON)