# Bridging Classical and Quantum Machine Learning: Knowledge Transfer From Classical to Quantum Neural Networks Using Knowledge Distillation

arXiv: https://arxiv.org/abs/2311.13810 · HTML: https://mlsanity.com/paper/2311.13810/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation.
- Builds on 26 of its 76 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Placed in

- Knowledge distillation / [Graph neural network distillation](https://mlsanity.com/distill/graph-neural-network-distillation/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [The power of quantum neural networks](https://mlsanity.com/paper/2011.00027/) (2020)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [Challenges and Opportunities in Quantum Machine Learning](https://mlsanity.com/paper/2303.09491/) (2023)
- [AdaGrad stepsizes: Sharp convergence over nonconvex landscapes](https://mlsanity.com/paper/1806.01811/) (2018)
- [Efficient Learning for Deep Quantum Neural Networks](https://mlsanity.com/paper/1902.10445/) (2019)
- [Transfer learning in hybrid classical-quantum neural networks](https://mlsanity.com/paper/1912.08278/) (2019)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [Differentiable Learning of Quantum Circuit Born Machine](https://mlsanity.com/paper/1804.04168/) (2018)
- [Variants of RMSProp and Adagrad with Logarithmic Regret Bounds](https://mlsanity.com/paper/1706.05507/) (2017)
- [Quantum Neural Networks: Concepts, Applications, and Challenges](https://mlsanity.com/paper/2108.01468/) (2021)
- [Classical-to-Quantum Transfer Learning for Spoken Command Recognition Based on Quantum Neural Networks](https://mlsanity.com/paper/2110.08689/) (2021)
- [Classical-to-quantum convolutional neural network transfer learning](https://mlsanity.com/paper/2208.14708/) (2022)
- [A Classical-Quantum Convolutional Neural Network for Detecting Pneumonia from Chest Radiographs](https://mlsanity.com/paper/2202.10452/) (2022)
- [Comparing concepts of quantum and classical neural network models for image classification task](https://mlsanity.com/paper/2108.08875/) (2021)

## Built on by (most cited first, 3 of 3)

- [CQ CNN: A Hybrid Classical Quantum Convolutional Neural Network for Alzheimer's Disease Detection Using Diffusion Generated and U Net Segmented 3D MRI](https://mlsanity.com/paper/2503.02345/) (2025)
- [From Foundation ECG Models to NISQ Learners: Distilling ECGFounder into a VQC Student](https://mlsanity.com/paper/2603.27269/) (2026)
- [Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation](https://mlsanity.com/paper/2606.30520/) (2026)

## Abstract (arXiv)

Quantum neural networks (QNNs), harnessing superposition and entanglement, have shown potential to surpass classical methods in complex learning tasks but remain limited by hardware constraints and noisy conditions. In this work, we present a novel framework for transferring knowledge from classical convolutional neural networks (CNNs) to QNNs via knowledge distillation, thereby reducing the need for resource intensive quantum training and error mitigation. We conduct extensive experiments using two parameterized quantum circuits (PQCs) with 4 and 8 qubits on MNIST, Fashion MNIST, and CIFAR10 datasets. The approach demonstrates consistent accuracy improvements attributed to distilled knowledge from larger classical networks. Through ablation studies, we systematically compare the effect of state of the art dimensionality reduction techniques fully connected layers, center cropping, principal component analysis, and pooling to compress high-dimensional image data prior to quantum encoding. Our findings reveal that fully connected layers retain the most salient features for QNN inference, thereby surpassing other down sampling approaches. Additionally, we examine state of the art data encoding methods (amplitude, angle, and qubit encoding) and identify amplitude encoding as the optimal strategy, yielding superior accuracy across all tested datasets and qubit configurations. Through computational analyses, we show that our distilled 4-qubit and 8-qubit QNNs achieve competitive performance while utilizing significantly fewer parameters than their classical counterparts. Our results establish a promising paradigm for bridging classical deep learning and emerging quantum computing, paving the way for more powerful, resource conscious models in quantum machine intelligence.