mlsanityData to 7 Oct 2026

Better than classical? The subtle art of benchmarking quantum machine learning models

arXiv 2403.07059 ↗First posted 11 Mar 2024quant-ph cs.LGcited by 82

Not on the map. Why?

Abstract (arXiv)

Benchmarking models via classical simulations is one of the main ways to judge ideas in quantum machine learning before noise-free hardware is available. However, the huge impact of the experimental design on the results, the small scales within reach today, as well as narratives influenced by the commercialisation of quantum technologies make it difficult to gain robust insights. To facilitate better decision-making we develop an open-source package based on the PennyLane software framework and use it to conduct a large-scale study that systematically tests 12 popular quantum machine learning models on 6 binary classification tasks used to create 160 individual datasets. We find that overall, out-of-the-box classical machine learning models outperform the quantum classifiers. Moreover, removing entanglement from a quantum model often results in as good or better performance, suggesting that "quantumness" may not be the crucial ingredient for the small learning tasks considered here. Our benchmarks also unlock investigations beyond simplistic leaderboard comparisons, and we identify five important questions for quantum model design that follow from our results.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 37 of its 103 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

37 of 103 references.

References · 37

  1. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  3. 2017Deep Reinforcement Learning that Matterscited by 727
  4. 2018Supervised learning with quantum enhanced feature spacescited by 654
  5. 2018Barren plateaus in quantum neural network training landscapescited by 624
  6. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  7. 2020Power of data in quantum machine learningcited by 336
  8. 2020The effect of data encoding on the expressive power of variational quantum machine learning modelscited by 328
  9. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  10. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  11. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  12. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
25 more
  1. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  2. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  3. 2021Supervised quantum machine learning models are kernel methodscited by 199
  4. 2018Do CIFAR-10 Classifiers Generalize to CIFAR-10?cited by 197
  5. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  6. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 164
  7. 2021The Benchmark Lotterycited by 133
  8. 2019Transfer learning in hybrid classical-quantum neural networkscited by 113
  9. 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 100
  10. 2021The Inductive Bias of Quantum Kernelscited by 90
  11. 2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
  12. 2020Recurrent Quantum Neural Networkscited by 79
  13. 2022Group-Invariant Quantum Machine Learningcited by 56
  14. 2022Classical surrogates for quantum learning modelscited by 39
  15. 2020Deep Networks and the Multiple Manifold Problemcited by 35
  16. 2023On quantum backpropagation, information reuse, and cheating measurement collapsecited by 32
  17. 2021Entangled Datasets for Quantum Machine Learningcited by 26
  18. 2019Value-laden Disciplinary Shifts in Machine Learningcited by 25
  19. 2020Scaling Down Deep Learning with MNIST-1Dcited by 23
  20. 2023ConvNets Match Vision Transformers at Scalecited by 16
  21. 2023An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-based Gradientscited by 14
  22. 2021A Novel Intrinsic Measure of Data Separabilitycited by 9
  23. 2023Symmetry-invariant quantum machine learning force fieldscited by 8
  24. 2022Towards Bundle Adjustment for Satellite Imaging via Quantum Machine Learningcited by 5
  25. 2023Application-Oriented Benchmarking of Quantum Generative Learning Using QUARKcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 25; 2026: 33.

82 papers.

Sort:

By area:

  1. 2024Quantum Architecture Search: A Surveycited by 31
  2. 2024Quantum Kernel Methods under Scrutiny: A Benchmarking Studycited by 22
  3. 2024Benchmarking quantum machine learning kernel training for classification taskscited by 10
  4. 2025Benchmarking Quantum Reinforcement Learningcited by 9
  5. 2025A Primer on Quantum Machine Learningcited by 9
  6. 2024Reinforcement learning-based architecture search for quantum machine learningcited by 8
  7. 2024Opportunities and limitations of explaining quantum machine learningcited by 6
  8. 2024Quantum Neural Networks in Practice: A Comparative Study with Classical Models from Standard Data Sets to Industrial Imagescited by 5
  9. 2025Data re-uploading in Quantum Machine Learning for time series: application to traffic forecastingcited by 5
  10. 2026Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computingcited by 5
  11. 2024Guided-SPSA: Simultaneous Perturbation Stochastic Approximation assisted by the Parameter Shift Rulecited by 4
  12. 2024QUACK: Quantum Aligned Centroid Kernelcited by 4
38 more
  1. 2024Quantum Hamiltonian Embedding of Images for Data Reuploading Classifierscited by 4
  2. 2025Leveraging Diffusion Models for Parameterized Quantum Circuit Generationcited by 4
  3. 2024Exploiting the equivalence between quantum neural networks and perceptronscited by 3
  4. 2025Stability and Generalization of Quantum Neural Networkscited by 3
  5. 2025On the similarity of bandwidth-tuned quantum kernels and classical kernelscited by 3
  6. 2025AI Benchmark Democratization and Carpentrycited by 3
  7. 2026MerLin: A Discovery Engine for Photonic and Hybrid Quantum Machine Learningcited by 3
  8. 2026Spectral methods: crucial for machine learning, natural for quantum computers?cited by 3
  9. 2024The Impact of Feature Embedding Placement in the Ansatz of a Quantum Kernel in QSVMscited by 2
  10. 2024Satellite image classification with neural quantum kernelscited by 2
  11. 2024Regression and Classification with Single-Qubit Quantum Neural Networkscited by 2
  12. 2025Multiple Embeddings for Quantum Machine Learningcited by 2
  13. 2025TabularQGAN: A quantum generative model for tabular data synthesiscited by 2
  14. 2025An MLCommons Scientific Benchmarks Ontologycited by 2
  15. 2025Learning Reduced Representations for Quantum Classifierscited by 2
  16. 2026Hybrid Classical-Quantum Transfer Learning with Noisy Quantum Circuitscited by 2
  17. 2026Non-variational supervised quantum kernel methods: a reviewcited by 2
  18. 2024On Optimizing Hyperparameters for Quantum Neural Networkscited by 1
  19. 2024Empowering Credit Scoring Systems with Quantum-Enhanced Machine Learningcited by 1
  20. 2024Hype or Heuristic? Quantum Reinforcement Learning for Join Order Optimisationcited by 1
  21. 2024MEMO-QCD: Quantum Density Estimation through Memetic Optimisation for Quantum Circuit Designcited by 1
  22. 2024Security Concerns in Quantum Machine Learning as a Servicecited by 1
  23. 2024Data Efficient Prediction of excited-state properties using Quantum Neural Networkscited by 1
  24. 2025Benchmarking Quantum Convolutional Neural Networks for Signal Classification in Simulated Gamma-Ray Burst Detectioncited by 1
  25. 2025Advances in Machine Learning: Where Can Quantum Techniques Help?cited by 1
  26. 2025Quantum feature-map learning with reduced resource overheadcited by 1
  27. 2025Quantum Deep Learning Still Needs a Quantum Leapcited by 1
  28. 2026The Role of Quantum in Hybrid Quantum-Classical Neural Networks: A Realistic Assessmentcited by 1
  29. 2026Q-ShiftDP: A Differentially Private Parameter-Shift Rule for Quantum Machine Learningcited by 1
  30. 2026Benchmarking Anomaly Detection Across Heterogeneous Cloud Telemetry Datasetscited by 1
  31. 2026Theory and interpretability of Quantum Extreme Learning Machines: a Pauli-transfer matrix approachcited by 1
  32. 2026On the Representational Limits of Quantum-Inspired 1024-D Document Embeddings: An Experimental Evaluation Frameworkcited by 1
  33. 2024Expanding the Horizon: Enabling Hybrid Quantum Transfer Learning for Long-Tailed Chest X-Ray Classificationcited by 0
  34. 2024Hybrid Classical-Quantum architecture for vectorised image classification of hand-written sketchescited by 0
  35. 2024Let the Quantum Creep In: Designing Quantum Neural Network Models by Gradually Swapping Out Classical Componentscited by 0
  36. 2024AQMLator -- An Auto Quantum Machine Learning E-Platformcited by 0
  37. 2024Computable Model-Independent Bounds for Adversarial Quantum Machine Learningcited by 0
  38. 2025AutoQML: A Framework for Automated Quantum Machine Learningcited by 0

50 most cited of 82 All 82 (JSON)