mlsanityData to 7 Oct 2026

Why is AI hard and Physics simple?

arXiv 2104.00008 ↗First posted 31 Mar 2021hep-th cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We discuss why AI is hard and why physics is simple. We discuss how physical intuition and the approach of theoretical physics can be brought to bear on the field of artificial intelligence and specifically machine learning. We suggest that the underlying project of machine learning and the underlying project of physics are strongly coupled through the principle of sparsity, and we call upon theoretical physicists to work on AI as physicists. As a first step in that direction, we discuss an upcoming book on the principles of deep learning theory that attempts to realize this approach.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 81 references that are in the map. First posted on arXiv 31 Mar 2021.

Builds on

19 of 81 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  8. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  9. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  12. 2017Deep Neural Networks as Gaussian Processescited by 617
7 more
  1. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  2. 2016On the Expressive Power of Deep Neural Networkscited by 302
  3. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  4. 2016Deep Information Propagationcited by 194
  5. 2019Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networkscited by 162
  6. 2017Building machines that adapt and compute like brainscited by 99
  7. 2019Non-Gaussian processes and neural networks at finite widthscited by 78

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 1; 2025: 1; 2026: 1.

7 papers.

  1. 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 121
  2. 2021The edge of chaos: quantum field theory and deep neural networkscited by 23
  3. 2024Neural Scaling Laws Rooted in the Data Distributioncited by 14
  4. 2023Structures of Neural Network Effective Theoriescited by 13
  5. 2022Laziness, Barren Plateau, and Noise in Machine Learningcited by 10
  6. 2025Gauge-covariant stochastic neural fields: Stability and finite-width effectscited by 2
  7. 2026TetrisCNN for interpretable detection of phases of matter from experimental quantum simulator datacited by 0