mlsanityData to 14 Oct 2026

Fisher Information, Training and Bias in Fourier Regression Models

arXiv 2510.06945 ↗First posted 8 Oct 2025cs.LG cond-mat.dis-nn physics.data-ancited by 0

Not on the map. Why?

Abstract (arXiv)

Motivated by the growing interest in quantum machine learning, in particular quantum neural networks (QNNs), we study how recently introduced evaluation metrics based on the Fisher information matrix (FIM) are effective for predicting their training and prediction performance. We exploit the equivalence between a broad class of QNNs and Fourier models, and study the interplay between the effective dimension and the bias of a model towards a given task, investigating how these affect the model's training and performance. We show that for a model that is completely agnostic, or unbiased, towards the function to be learned, a higher effective dimension likely results in a better trainability and performance. On the other hand, for models that are biased towards the function to be learned a lower effective dimension is likely beneficial during training. To obtain these results, we derive an analytical expression of the FIM for Fourier models and identify the features controlling a model's effective dimension. This allows us to construct models with tunable effective dimension and bias, and to compare their training. We furthermore introduce a tensor network representation of the considered Fourier models, which could be a tool of independent interest for the analysis of QNN models. Overall, these findings provide an explicit example of the interplay between geometrical properties, model-task alignment and training, which are relevant for the broader machine learning community.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 78 references that are in the map. First posted on arXiv 8 Oct 2025.

Builds on

32 of 78 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2020Variational Quantum Algorithmscited by 692
  4. 2020The power of quantum neural networkscited by 379
  5. 2020The effect of data encoding on the expressive power of variational quantum machine learning modelscited by 333
  6. 2019Parameterized quantum circuits as machine learning modelscited by 332
  7. 2021Generalization in quantum machine learning from few training datacited by 216
  8. 2023Challenges and Opportunities in Quantum Machine Learningcited by 203
  9. 2021Supervised quantum machine learning models are kernel methodscited by 200
  10. 2020Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernelcited by 187
  11. 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
  12. 2013Revisiting Natural Gradient for Deep Networkscited by 159
20 more
  1. 2021Theory of overparametrization in quantum neural networkscited by 100
  2. 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 91
  3. 2021The Inductive Bias of Quantum Kernelscited by 91
  4. 2021Encoding-dependent generalization bounds for parametrized quantum circuitscited by 82
  5. 2020Quantum Convolutional Neural Networks for High Energy Physics Data Analysiscited by 64
  6. 2021A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learningcited by 57
  7. 2022Classical surrogates for quantum learning modelscited by 39
  8. 2019Input Redundancy for Parameterized Quantum Circuitscited by 33
  9. 2019Pathological spectra of the Fisher information metric and its variants in deep neural networkscited by 32
  10. 2018Fisher Information and Natural Gradient Learning of Random Deep Networkscited by 28
  11. 2021A Theory of Neural Tangent Kernel Alignment and Its Influence on Trainingcited by 25
  12. 2023Quantum anomaly detection in the latent space of proton collision events at the LHCcited by 21
  13. 2021Effective dimension of machine learning modelscited by 14
  14. 2023Generalization of Quantum Machine Learning Models Using Quantum Fisher Information Metriccited by 10
  15. 2020The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometrycited by 9
  16. 2024Classification of the Fashion-MNIST Dataset on a Quantum Computercited by 9
  17. 2024Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacitycited by 9
  18. 2024Quantum Large Language Models via Tensor Network Disentanglerscited by 7
  19. 2023Hybrid Quantum Machine Learning Assisted Classification of COVID-19 from Computed Tomography Scanscited by 4
  20. 2024Building Continuous Quantum-Classical Bayesian Neural Networks for a Classical Clinical Datasetcited by 4

Built on by

Not cited yet.