mlsanityData to 14 Oct 2026

Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel

arXiv 2202.05254 ↗First posted 10 Feb 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A biological neural network in the cortex forms a neural field. Neurons in the field have their own receptive fields, and connection weights between two neurons are random but highly correlated when they are in close proximity in receptive fields. In this paper, we investigate such neural fields in a multilayer architecture to investigate the supervised learning of the fields. We empirically compare the performances of our field model with those of randomly connected deep networks. The behavior of a randomly connected network is investigated on the basis of the key idea of the neural tangent kernel regime, a recent development in the machine learning theory of over-parameterized networks; for most randomly connected neural networks, it is shown that global minima always exist in their small neighborhoods. We numerically show that this claim also holds for our neural fields. In more detail, our model has two structures: i) each neuron in a field has a continuously distributed receptive field, and ii) the initial connection weights are random but not independent, having correlations when the positions of neurons are close in each layer. We show that such a multilayer neural field is more robust than conventional models when input patterns are deformed by noise disturbances. Moreover, its generalization ability can be slightly superior to that of conventional models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 54 references that are in the map. First posted on arXiv 10 Feb 2022.

Builds on

22 of 54 references.

References · 22

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  3. 2018On Lazy Training in Differentiable Programmingcited by 675
  4. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  5. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  6. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  7. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  8. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  9. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  10. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 261
  11. 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 222
  12. 2016Deep Information Propagationcited by 195
10 more
  1. 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
  2. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 167
  3. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 149
  4. 2018Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networkscited by 141
  5. 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 91
  6. 2020Mat'ern Gaussian processes on Riemannian manifoldscited by 82
  7. 2020Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHScited by 74
  8. 2016Diverse Neural Network Learns True Target Functionscited by 33
  9. 2019Pathological spectra of the Fisher information metric and its variants in deep neural networkscited by 32
  10. 2020Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspectivecited by 5

Built on by

Not cited yet.