Generalisation and benign over-fitting for linear regression onto random functional covariates

We study theoretical predictive performance of ridge and ridge-less least-squares regression when covariate vectors arise from evaluating p random, means-square continuous functions over a latent metric space at n random and unobserved locations, subject to additive noise. This leads us away from the standard assumption of i.i.d. data to a setting in which the n covariate vectors are exchangeable but not independent in general. Under an assumption of independence across dimensions, 4-th order moment, and other regularity conditions, we obtain probabilistic bounds on a notion of predictive excess risk adapted to our random functional covariate setting, making use of recent results of Barzilai and Shamir. We derive convergence rates in regimes where p grows suitably fast relative to n, illustrating interplay between ingredients of the model in determining convergence behaviour and the role of additive covariate noise in benign-overfitting.