mlsanityData to 7 Oct 2026

What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement

arXiv 2303.11249 ↗First posted 20 Mar 2023cs.LG cs.AI quant-phcited by 4

Not on the map. Why?

Abstract (arXiv)

The question of what makes a data distribution suitable for deep learning is a fundamental open problem. Focusing on locally connected neural networks (a prevalent family of architectures that includes convolutional and recurrent neural networks as well as local self-attention models), we address this problem by adopting theoretical tools from quantum physics. Our main theoretical result states that a certain locally connected neural network is capable of accurate prediction over a data distribution if and only if the data distribution admits low quantum entanglement under certain canonical partitions of features. As a practical application of this result, we derive a preprocessing method for enhancing the suitability of a data distribution to locally connected neural networks. Experiments with widespread models over various datasets demonstrate our findings. We hope that our use of quantum entanglement will encourage further adoption of tools from physics for formally reasoning about the relation between deep learning and real-world data.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 72 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

46 of 72 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  4. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  5. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  6. 2016Error bounds for approximations with deep ReLU networkscited by 567
  7. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  8. 2014OpenML: networked science in machine learningcited by 523
  9. 2020Implicit Regularization in Deep Learning May Not Be Explainable by Normscited by 118
  10. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  11. 2017Failures of Gradient-Based Deep Learningcited by 79
  12. 2016Very Deep Convolutional Neural Networks for Raw Waveformscited by 68
34 more
  1. 2017Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputscited by 61
  2. 2016Inductive Bias of Deep Convolutional Networks through Pooling Geometrycited by 53
  3. 2017Learning Relevant Features of Data with Multi-scale Tensor Networkscited by 50
  4. 2018Quantum Entanglement in Deep Learning Architecturescited by 49
  5. 2021Implicit Regularization in Tensor Factorizationcited by 43
  6. 2017Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minimacited by 40
  7. 2018Exponential Convergence of the Deep Neural Network Approximation for Analytic Functionscited by 38
  8. 2017Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machinescited by 33
  9. 2016Convolutional Rectifier Networks as Generalized Tensor Decompositionscited by 32
  10. 2017Expressive power of recurrent neural networkscited by 30
  11. 2020Quantum-inspired Machine Learning on high-energy physics datacited by 30
  12. 2022Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networkscited by 27
  13. 2020Do Transformers Need Deep Long-Range Memorycited by 24
  14. 2017Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Designcited by 22
  15. 2021Understanding Deflation Process in Over-parametrized Tensor Decompositioncited by 20
  16. 2018A Provably Correct Algorithm for Deep Learning that Actually Workscited by 19
  17. 2020Entanglement and Tensor Networks for Supervised Image Classificationcited by 18
  18. 2020Beyond Lazy Training for Over-parameterized Tensor Decompositioncited by 15
  19. 2017On the Long-Term Memory of Deep Recurrent Networkscited by 14
  20. 2017Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositionscited by 13
  21. 2020Understanding Generalization in Deep Learning via Tensor Methodscited by 13
  22. 2021Tensor networks and efficient descriptions of classical datacited by 13
  23. 2022On the Ability of Graph Neural Networks to Model Interactions Between Verticescited by 12
  24. 2015Deep SimNetscited by 10
  25. 2019Generalized Tensor Models for Recurrent Neural Networkscited by 8
  26. 2018End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decompositioncited by 6
  27. 2018Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlapscited by 6
  28. 2014SimNets: A Generalization of Convolutional Networkscited by 5
  29. 2017Boosting Dilated Convolutional Networks with Mixed Tensor Decompositionscited by 5
  30. 2017Entanglement Entropy of Target Functions for Image Classification and Convolutional Neural Networkcited by 5
  31. 2022Empirical Evaluation and Theoretical Analysis for Representation Learning: A Surveycited by 5
  32. 2018Provable limitations of deep learningcited by 4
  33. 2021Entangled q-Convolutional Neural Netscited by 4
  34. 2021Tensor network to learn the wavefunction of datacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2026Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Datacited by 5
  2. 2024A Tensor Decomposition Perspective on Second-order RNNscited by 3
  3. 2024Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learningcited by 1
  4. 2026On the Role of Depth in the Expressivity of RNNscited by 0