# Transformer-QEC: Quantum Error Correction Code Decoding with Transferable Transformers

arXiv: https://arxiv.org/abs/2311.16082 · HTML: https://mlsanity.com/paper/2311.16082/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning](https://mlsanity.com/paper/2012.09852/) (2020)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [SpArch: Efficient Architecture for Sparse Matrix Multiplication](https://mlsanity.com/paper/2002.08947/) (2020)
- [Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation](https://mlsanity.com/paper/1810.07207/) (2018)
- [QuantumNAT: Quantum Noise-Aware Training with Noise Injection, Quantization and Normalization](https://mlsanity.com/paper/2110.11331/) (2021)
- [PointAcc: Efficient Point Cloud Accelerator](https://mlsanity.com/paper/2110.07600/) (2021)
- [QOC: Quantum On-Chip Training with Parameter Shift and Gradient Pruning](https://mlsanity.com/paper/2202.13239/) (2022)
- [Deep neural decoders for near term fault-tolerant experiments](https://mlsanity.com/paper/1802.06441/) (2018)
- [Can Noise on Qubits Be Learned in Quantum Neural Network? A Case Study on QuantumFlow](https://mlsanity.com/paper/2109.03430/) (2021)
- [SnCQA: A hardware-efficient equivariant quantum convolutional circuit architecture](https://mlsanity.com/paper/2211.12711/) (2022)
- [Hybrid Gate-Pulse Model for Variational Quantum Algorithms](https://mlsanity.com/paper/2212.00661/) (2022)
- [Fusion Blossom: Fast MWPM Decoders for QEC](https://mlsanity.com/paper/2305.08307/) (2023)
- [TopGen: Topology-Aware Bottom-Up Generator for Variational Quantum Circuits](https://mlsanity.com/paper/2210.08190/) (2022)
- [LILLIPUT: A Lightweight Low-Latency Lookup-Table Based Decoder for Near-term Quantum Error Correction](https://mlsanity.com/paper/2108.06569/) (2021)

## Built on by (most cited first, 10 of 10)

- [Artificial intelligence for representing and characterizing quantum systems](https://mlsanity.com/paper/2509.04923/) (2025)
- [Toward Uncertainty-Aware and Generalizable Neural Decoding for Quantum LDPC Codes](https://mlsanity.com/paper/2510.06257/) (2025)
- [Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning](https://mlsanity.com/paper/2408.09524/) (2024)
- [A Symmetry-Integrated Approach to Surface Code Decoding](https://mlsanity.com/paper/2509.10164/) (2025)
- [Dynamic Inhomogeneous Quantum Resource Scheduling with Reinforcement Learning](https://mlsanity.com/paper/2405.16380/) (2024)
- [Autoregressive model path dependence near Ising criticality](https://mlsanity.com/paper/2408.15715/) (2024)
- [Qubit-centric Transformer for Surface Code Decoding](https://mlsanity.com/paper/2510.11593/) (2025)
- [QGEC : Quantum Golay Code Error Correction](https://mlsanity.com/paper/2512.11307/) (2025)
- [A Mixture of Experts Vision Transformer for High-Fidelity Surface Code Decoding](https://mlsanity.com/paper/2601.12483/) (2026)
- [When AI meets quantum information: A comprehensive review](https://mlsanity.com/paper/2607.00365/) (2026)

## Abstract (arXiv)

Quantum computing has the potential to solve problems that are intractable for classical systems, yet the high error rates in contemporary quantum devices often exceed tolerable limits for useful algorithm execution. Quantum Error Correction (QEC) mitigates this by employing redundancy, distributing quantum information across multiple data qubits and utilizing syndrome qubits to monitor their states for errors. The syndromes are subsequently interpreted by a decoding algorithm to identify and correct errors in the data qubits. This task is complex due to the multiplicity of error sources affecting both data and syndrome qubits as well as syndrome extraction operations. Additionally, identical syndromes can emanate from different error sources, necessitating a decoding algorithm that evaluates syndromes collectively. Although machine learning (ML) decoders such as multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) have been proposed, they often focus on local syndrome regions and require retraining when adjusting for different code distances. We introduce a transformer-based QEC decoder which employs self-attention to achieve a global receptive field across all input syndromes. It incorporates a mixed loss training approach, combining both local physical error and global parity label losses. Moreover, the transformer architecture's inherent adaptability to variable-length inputs allows for efficient transfer learning, enabling the decoder to adapt to varying code distances without retraining. Evaluation on six code distances and ten different error configurations demonstrates that our model consistently outperforms non-ML decoders, such as Union Find (UF) and Minimum Weight Perfect Matching (MWPM), and other ML decoders, thereby achieving best logical error rates. Moreover, the transfer learning can save over 10x of training cost.