# qecGPT: decoding Quantum Error-correcting Codes with Generative Pre-trained Transformers

arXiv: https://arxiv.org/abs/2307.09025 · HTML: https://mlsanity.com/paper/2307.09025/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 29 references that are in the map.
- First posted on arXiv 18 Jul 2023.

## Builds on (most cited first, 4 of 4)

- [Solving Statistical Mechanics Using Variational Autoregressive Networks](https://mlsanity.com/paper/1809.10606/) (2018)
- [TraDE: Transformers for Density Estimation](https://mlsanity.com/paper/2004.02441/) (2020)
- [Scalable detection of statistically significant communities and hierarchies, using message-passing for modularity](https://mlsanity.com/paper/1403.5787/) (2014)
- [A scalable and fast artificial neural network syndrome decoder for surface codes](https://mlsanity.com/paper/2110.05854/) (2021)

## Built on by (most cited first, 9 of 9)

- [Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network](https://mlsanity.com/paper/2310.05900/) (2023)
- [Artificial intelligence for representing and characterizing quantum systems](https://mlsanity.com/paper/2509.04923/) (2025)
- [Efficient and Universal Neural-Network Decoder for Stabilizer-Based Quantum Error Correction](https://mlsanity.com/paper/2502.19971/) (2025)
- [Scalable Neural Decoders for Practical Real-Time Quantum Error Correction](https://mlsanity.com/paper/2510.22724/) (2025)
- [Rethink the Role of Neural Decoders in Quantum Error Correction](https://mlsanity.com/paper/2605.12046/) (2026)
- [Maximum Likelihood Decoding of Quantum Error Correction Codes](https://mlsanity.com/paper/2605.17230/) (2026)
- [Efficient foundation decoders for fault-tolerant quantum computing](https://mlsanity.com/paper/2606.27119/) (2026)
- [Machine Learning Approaches to Decoding Topological Quantum Codes](https://mlsanity.com/paper/2608.15760/) (2026)
- [When Quantum Meets AI: Quantum Methods for Machine Learning and Machine Learning Methods for Quantum Systems](https://mlsanity.com/paper/2609.25641/) (2026)

## Abstract (arXiv)

We propose a general framework for decoding quantum error-correcting codes with generative modeling. The model utilizes autoregressive neural networks, specifically Transformers, to learn the joint probability of logical operators and syndromes. This training is in an unsupervised way, without the need for labeled training data, and is thus referred to as pre-training. After the pre-training, the model can efficiently compute the likelihood of logical operators for any given syndrome, using maximum likelihood decoding. It can directly generate the most-likely logical operators with computational complexity mathcal O(2k) in the number of logical qubits k, which is significantly better than the conventional maximum likelihood decoding algorithms that require mathcal O(4k) computation. Based on the pre-trained model, we further propose refinement to achieve more accurately the likelihood of logical operators for a given syndrome by directly sampling the stabilizer operators. We perform numerical experiments on stabilizer codes with small code distances, using both depolarizing error models and error models with correlated noise. The results show that our approach provides significantly better decoding accuracy than the minimum weight perfect matching and belief-propagation-based algorithms. Our framework is general and can be applied to any error model and quantum codes with different topologies such as surface codes and quantum LDPC codes. Furthermore, it leverages the parallelization capabilities of GPUs, enabling simultaneous decoding of a large number of syndromes. Our approach sheds light on the efficient and accurate decoding of quantum error-correcting codes using generative artificial intelligence and modern computational power.