mlsanityData to 7 Oct 2026

On the inability of Gaussian process regression to optimally learn compositional functions

arXiv 2205.07764 ↗First posted 16 May 2022stat.ML cs.LG math.STcited by 7

Not on the map. Why?

Abstract (arXiv)

We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 61 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

19 of 61 references.

References · 19

  1. 2016Error bounds for approximations with deep ReLU networkscited by 567
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2016Benefits of depth in neural networkscited by 248
  4. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  5. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  6. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  7. 2020The Kolmogorov-Arnold representation theorem revisitedcited by 89
  8. 2017Frequentist Consistency of Variational Bayescited by 60
  9. 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
  10. 2018Posterior Concentration for Sparse Deep Learningcited by 50
  11. 2019Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov spacecited by 46
  12. 2019Variational Bayes for high-dimensional linear regression with sparse priorscited by 44
7 more
  1. 2019Convergence Rates of Variational Inference in Sparse Deep Learningcited by 27
  2. 2019On the minimax optimality and superiority of deep neural network learning over sparse parameter spacescited by 24
  3. 2019Variational Bayes under Model Misspecificationcited by 22
  4. 2020Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfacescited by 17
  5. 2020Spike and slab variational Bayes for high dimensional logistic regressioncited by 16
  6. 2020Estimation error analysis of deep learning on the regression problem on the variable exponent Besov spacecited by 13
  7. 2014Frequentist and Bayesian inference for Gaussian-log-Gaussian wavelet trees, and statistical signal processing applicationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 2; 2026: 1.

7 papers.

  1. 2022What Can Be Learnt With Wide Convolutional Neural Networks?cited by 16
  2. 2024How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learningcited by 6
  3. 2024Deep Horseshoe Gaussian Processescited by 5
  4. 2026PI-JEPA: Label-Free Surrogate Pretraining for Coupled Multiphysics Simulation via Operator-Split Latent Predictioncited by 3
  5. 2023Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learningcited by 1
  6. 2025Preference Learning with Response Time: Robust Losses and Guaranteescited by 1
  7. 2025Support Collapse of Deep Gaussian Processes with Polynomial Kernels for a Wide Regime of Hyperparameterscited by 0