# Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network

arXiv: https://arxiv.org/abs/2310.05900 · HTML: https://mlsanity.com/paper/2310.05900/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 133 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Large Batch Optimization for Deep Learning: Training BERT in 76 minutes](https://mlsanity.com/paper/1904.00962/) (2019)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [Power of data in quantum machine learning](https://mlsanity.com/paper/2011.01938/) (2020)
- [Quantum advantage in learning from experiments](https://mlsanity.com/paper/2112.00778/) (2021)
- [Recurrent Quantum Neural Networks](https://mlsanity.com/paper/2006.14619/) (2020)
- [Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation](https://mlsanity.com/paper/1810.07207/) (2018)
- [Deep neural decoders for near term fault-tolerant experiments](https://mlsanity.com/paper/1802.06441/) (2018)
- [Deep Quantum Error Correction](https://mlsanity.com/paper/2301.11930/) (2023)
- [Advantages of versatile neural-network decoding for topological codes](https://mlsanity.com/paper/1802.08680/) (2018)
- [qecGPT: decoding Quantum Error-correcting Codes with Generative Pre-trained Transformers](https://mlsanity.com/paper/2307.09025/) (2023)
- [Neural Network Decoders for Large-Distance 2D Toric Codes](https://mlsanity.com/paper/1809.06640/) (2018)
- [A scalable and fast artificial neural network syndrome decoder for surface codes](https://mlsanity.com/paper/2110.05854/) (2021)
- [The END: An Equivariant Neural Decoder for Quantum Error Correction](https://mlsanity.com/paper/2304.07362/) (2023)
- [Learnability and Complexity of Quantum Samples](https://mlsanity.com/paper/2010.11983/) (2020)
- [A Scalable, Fast and Programmable Neural Decoder for Fault-Tolerant Quantum Computation Using Surface Codes](https://mlsanity.com/paper/2305.15767/) (2023)
- [Scalable Quantum Error Correction for Surface Codes using FPGA](https://mlsanity.com/paper/2301.08419/) (2023)
- [Decoding surface codes with deep reinforcement learning and probabilistic policy reuse](https://mlsanity.com/paper/2212.11890/) (2022)

## Built on by (most cited first, 30 of 42)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Qiskit Code Assistant: Training LLMs for generating Quantum Computing Code](https://mlsanity.com/paper/2405.19495/) (2024)
- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [A Primer on Quantum Machine Learning](https://mlsanity.com/paper/2511.15969/) (2025)
- [Artificial intelligence for representing and characterizing quantum systems](https://mlsanity.com/paper/2509.04923/) (2025)
- [A scalable and real-time neural decoder for topological quantum codes](https://mlsanity.com/paper/2512.07737/) (2025)
- [Efficient and Universal Neural-Network Decoder for Stabilizer-Based Quantum Error Correction](https://mlsanity.com/paper/2502.19971/) (2025)
- [Quantum computing and artificial intelligence: status and perspectives](https://mlsanity.com/paper/2505.23860/) (2025)
- [MetaSym: A Symplectic Meta-learning Framework for Physical Intelligence](https://mlsanity.com/paper/2502.16667/) (2025)
- [Scalable Neural Decoders for Practical Fault-Tolerant Quantum Computation](https://mlsanity.com/paper/2604.08358/) (2026)
- [Scalable Neural Decoders for Practical Real-Time Quantum Error Correction](https://mlsanity.com/paper/2510.22724/) (2025)
- [Rethink the Role of Neural Decoders in Quantum Error Correction](https://mlsanity.com/paper/2605.12046/) (2026)
- [Learning to Decode in Parallel: Self-Coordinating Neural Network for Real-Time Quantum Error Correction](https://mlsanity.com/paper/2601.09921/) (2026)
- [Learning Encodings by Maximizing State Distinguishability: Variational Quantum Error Correction](https://mlsanity.com/paper/2506.11552/) (2025)
- [Embedding-Aware Quantum-Classical SVMs for Scalable Quantum Machine Learning](https://mlsanity.com/paper/2508.00024/) (2025)
- [Neural Minimum Weight Perfect Matching for Quantum Error Codes](https://mlsanity.com/paper/2601.00242/) (2026)
- [Universal computation is intrinsic to language model decoding](https://mlsanity.com/paper/2601.08061/) (2026)
- [Maximum Likelihood Decoding of Quantum Error Correction Codes](https://mlsanity.com/paper/2605.17230/) (2026)
- [Efficient foundation decoders for fault-tolerant quantum computing](https://mlsanity.com/paper/2606.27119/) (2026)
- [Machine Learning Approaches to Decoding Topological Quantum Codes](https://mlsanity.com/paper/2608.15760/) (2026)
- [Autoregressive model path dependence near Ising criticality](https://mlsanity.com/paper/2408.15715/) (2024)
- [Noise tolerance via reinforcement: Learning a reinforced quantum dynamics](https://mlsanity.com/paper/2506.12418/) (2025)
- [Parity Cross-Resonance: A Multiqubit Gate](https://mlsanity.com/paper/2508.10807/) (2025)
- [Qubit-centric Transformer for Surface Code Decoding](https://mlsanity.com/paper/2510.11593/) (2025)
- [Data-Driven Learnability Transition of Measurement-Induced Entanglement](https://mlsanity.com/paper/2512.01317/) (2025)
- [SAQ: Stabilizer-Aware Quantum Error Correction Decoder](https://mlsanity.com/paper/2512.08914/) (2025)
- [Evolutionary BP+OSD Decoding for Low-Latency Quantum Error Correction](https://mlsanity.com/paper/2512.18273/) (2025)
- [A Mixture of Experts Vision Transformer for High-Fidelity Surface Code Decoding](https://mlsanity.com/paper/2601.12483/) (2026)
- [Trapped by simplicity: When Transformers fail to learn from noisy features](https://mlsanity.com/paper/2602.08695/) (2026)
- [O(n) alternative to Quantum Fourier Transform with efficient neural net classical post-processing](https://mlsanity.com/paper/2605.16998/) (2026)

## Abstract (arXiv)

Quantum error-correction is a prerequisite for reliable quantum computation. Towards this goal, we present a recurrent, transformer-based neural network which learns to decode the surface code, the leading quantum error-correction code. Our decoder outperforms state-of-the-art algorithmic decoders on real-world data from Google's Sycamore quantum processor for distance 3 and 5 surface codes. On distances up to 11, the decoder maintains its advantage on simulated data with realistic noise including cross-talk, leakage, and analog readout signals, and sustains its accuracy far beyond the 25 cycles it was trained on. Our work illustrates the ability of machine learning to go beyond human-designed algorithms by learning from data directly, highlighting machine learning as a strong contender for decoding in quantum computers.