# Efficient foundation decoders for fault-tolerant quantum computing

arXiv: https://arxiv.org/abs/2606.27119 · HTML: https://mlsanity.com/paper/2606.27119/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 91 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Net2Net: Accelerating Learning via Knowledge Transfer](https://mlsanity.com/paper/1511.05641/) (2015)
- [Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network](https://mlsanity.com/paper/2310.05900/) (2023)
- [STEM: Scaling Transformers with Embedding Modules](https://mlsanity.com/paper/2601.10639/) (2026)
- [Artificial intelligence for representing and characterizing quantum systems](https://mlsanity.com/paper/2509.04923/) (2025)
- [qecGPT: decoding Quantum Error-correcting Codes with Generative Pre-trained Transformers](https://mlsanity.com/paper/2307.09025/) (2023)
- [A scalable and real-time neural decoder for topological quantum codes](https://mlsanity.com/paper/2512.07737/) (2025)
- [Neural Network Decoders for Large-Distance 2D Toric Codes](https://mlsanity.com/paper/1809.06640/) (2018)
- [Efficient and Universal Neural-Network Decoder for Stabilizer-Based Quantum Error Correction](https://mlsanity.com/paper/2502.19971/) (2025)
- [Fusion Blossom: Fast MWPM Decoders for QEC](https://mlsanity.com/paper/2305.08307/) (2023)
- [Scalable Neural Decoders for Practical Fault-Tolerant Quantum Computation](https://mlsanity.com/paper/2604.08358/) (2026)
- [Rethink the Role of Neural Decoders in Quantum Error Correction](https://mlsanity.com/paper/2605.12046/) (2026)
- [Toward Uncertainty-Aware and Generalizable Neural Decoding for Quantum LDPC Codes](https://mlsanity.com/paper/2510.06257/) (2025)
- [HyperNQ: A Hypergraph Neural Network Decoder for Quantum LDPC Codes](https://mlsanity.com/paper/2511.01741/) (2025)
- [Maximum Likelihood Decoding of Quantum Error Correction Codes](https://mlsanity.com/paper/2605.17230/) (2026)

## Built on by (most cited first, 1 of 1)

- [OmniQEC: discovering practical quantum error-correcting codes by an AI scientist](https://mlsanity.com/paper/2607.25865/) (2026)

## Abstract (arXiv)

Foundation decoders, a class of high-capacity neural decoders, are leading candidates for fault-tolerant quantum computing, with accurate and efficient decoding at large code distances. However, their construction often faces a steep scaling barrier, as larger code distances rapidly amplify the cost of syndrome generation and neural optimization. To address this bottleneck, here we devise neural transfer unification (NTU), a unified framework for efficient foundation decoders. A central feature of NTU is its ability to align decoding tasks across code distances via algebraic structures shared by scalable code families, which enables knowledge learned on smaller codes to accelerate large-scale decoder training. We instantiate NTU as NTU-Transformer, a transformer-based neural decoder tailored for planar surface codes and bivariate bicycle codes. For planar surface codes under circuit-level noise, NTU-Transformer outperforms correlation-aware matching on the [![361,1,19]!] code and further scales to the [![625,1,25]!] code, where it exceeds standard matching through transfer adaptation. For the bivariate bicycle code with [![72,12,6]!], it surpasses Relay-BP in the low-physical-error regime. These results establish our proposal as a scalable route to amortized cross-distance training of foundation decoders for fault-tolerant quantum processors.