# Laziness, Barren Plateau, and Noise in Machine Learning

arXiv: https://arxiv.org/abs/2206.09313 · HTML: https://mlsanity.com/paper/2206.09313/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 71 references that are in the map.
- First posted on arXiv 19 Jun 2022.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [Barren plateaus in quantum neural network training landscapes](https://mlsanity.com/paper/1803.11173/) (2018)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [On Exact Computation with an Infinitely Wide Neural Net](https://mlsanity.com/paper/1904.11955/) (2019)
- [A rigorous and robust quantum speed-up in supervised machine learning](https://mlsanity.com/paper/2010.02174/) (2020)
- [Noise-Induced Barren Plateaus in Variational Quantum Algorithms](https://mlsanity.com/paper/2007.14384/) (2020)
- [Feature Learning in Infinite-Width Neural Networks](https://mlsanity.com/paper/2011.14522/) (2020)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks](https://mlsanity.com/paper/2006.13198/) (2020)
- [Absence of Barren Plateaus in Quantum Convolutional Neural Networks](https://mlsanity.com/paper/2011.02966/) (2020)
- [Theory of overparametrization in quantum neural networks](https://mlsanity.com/paper/2109.11676/) (2021)
- [Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes](https://mlsanity.com/paper/1706.10239/) (2017)
- [Effect of barren plateaus on gradient-free optimization](https://mlsanity.com/paper/2011.12245/) (2020)
- [Non-Gaussian processes and neural networks at finite widths](https://mlsanity.com/paper/1910.00019/) (2019)
- [Asymptotics of Wide Networks from Feynman Diagrams](https://mlsanity.com/paper/1909.11304/) (2019)
- [Neural Networks and Quantum Field Theory](https://mlsanity.com/paper/2008.08601/) (2020)
- [Representation Learning via Quantum Neural Tangent Kernels](https://mlsanity.com/paper/2111.04225/) (2021)
- [On the infinite width limit of neural networks with a standard parameterization](https://mlsanity.com/paper/2001.07301/) (2020)
- [Quantum Deep Learning](https://mlsanity.com/paper/1412.3489/) (2014)
- [Are wider nets better given the same number of parameters?](https://mlsanity.com/paper/2010.14495/) (2020)
- [Analytic theory for the dynamics of wide quantum neural networks](https://mlsanity.com/paper/2203.16711/) (2022)
- [A Convergence Theory for Over-parameterized Variational Quantum Eigensolvers](https://mlsanity.com/paper/2205.12481/) (2022)
- [Quantum tangent kernel](https://mlsanity.com/paper/2111.02951/) (2021)
- [Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning](https://mlsanity.com/paper/1904.03673/) (2019)

## Built on by (most cited first, 10 of 10)

- [Barren Plateaus in Variational Quantum Computing](https://mlsanity.com/paper/2405.00781/) (2024)
- [Symmetric Pruning in Quantum Neural Networks](https://mlsanity.com/paper/2208.14057/) (2022)
- [Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification](https://mlsanity.com/paper/2301.01597/) (2022)
- [Stochastic noise can be helpful for variational quantum algorithms](https://mlsanity.com/paper/2210.06723/) (2022)
- [Investigating and Mitigating Barren Plateaus in Variational Quantum Circuits: A Survey](https://mlsanity.com/paper/2407.17706/) (2024)
- [SnCQA: A hardware-efficient equivariant quantum convolutional circuit architecture](https://mlsanity.com/paper/2211.12711/) (2022)
- [The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning](https://mlsanity.com/paper/2212.11826/) (2022)
- [Dynamical transition in controllable quantum neural networks with large depth](https://mlsanity.com/paper/2311.18144/) (2023)
- [The curse of random quantum data](https://mlsanity.com/paper/2408.09937/) (2024)
- [Quantum-data-driven dynamical transition in quantum learning](https://mlsanity.com/paper/2410.01955/) (2024)

## Abstract (arXiv)

We define laziness to describe a large suppression of variational parameter updates for neural networks, classical or quantum. In the quantum case, the suppression is exponential in the number of qubits for randomized variational quantum circuits. We discuss the difference between laziness and barren plateau in quantum machine learning created by quantum physicists in citemcclean2018barren for the flatness of the loss function landscape during gradient descent. We address a novel theoretical understanding of those two phenomena in light of the theory of neural tangent kernels. For noiseless quantum circuits, without the measurement noise, the loss function landscape is complicated in the overparametrized regime with a large number of trainable variational angles. Instead, around a random starting point in optimization, there are large numbers of local minima that are good enough and could minimize the mean square loss function, where we still have quantum laziness, but we do not have barren plateaus. However, the complicated landscape is not visible within a limited number of iterations, and low precision in quantum control and quantum sensing. Moreover, we look at the effect of noises during optimization by assuming intuitive noise models, and show that variational quantum algorithms are noise-resilient in the overparametrization regime. Our work precisely reformulates the quantum barren plateau statement towards a precision statement and justifies the statement in certain noise models, injects new hope toward near-term variational quantum algorithms, and provides theoretical connections toward classical machine learning. Our paper provides conceptual perspectives about quantum barren plateaus, together with discussions about the gradient descent dynamics in citetogether.