# Deepcode: Feedback Codes via Deep Learning

arXiv: https://arxiv.org/abs/1807.00801 · HTML: https://mlsanity.com/paper/1807.00801/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2018.

## Built on by (most cited first, 30 of 42)

- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation](https://mlsanity.com/paper/2502.19166/) (2025)
- [Autoregressive Belief Propagation for Decoding Block Codes](https://mlsanity.com/paper/2103.11780/) (2021)
- [ProductAE: Towards Training Larger Channel Codes based on Neural Product Codes](https://mlsanity.com/paper/2110.04466/) (2021)
- [Cyclically Equivariant Neural Decoders for Cyclic Codes](https://mlsanity.com/paper/2105.05540/) (2021)
- [Domain Generalization in Machine Learning Models for Wireless Communications: Concepts, State-of-the-Art, and Open Issues](https://mlsanity.com/paper/2303.08106/) (2023)
- [Deep Extended Feedback Codes](https://mlsanity.com/paper/2105.01365/) (2021)
- [All you need is feedback: Communication with block attention feedback codes](https://mlsanity.com/paper/2206.09457/) (2022)
- [Robust Bayesian Learning for Reliable Wireless AI: Framework and Applications](https://mlsanity.com/paper/2207.00300/) (2022)
- [Learning Linear Block Error Correction Codes](https://mlsanity.com/paper/2405.04050/) (2024)
- [AttentionCode: Ultra-Reliable Feedback Codes for Short-Packet Communications](https://mlsanity.com/paper/2205.14955/) (2022)
- [A Channel Coding Benchmark for Meta-Learning](https://mlsanity.com/paper/2107.07579/) (2021)
- [DRF Codes: Deep SNR-Robust Feedback Codes](https://mlsanity.com/paper/2112.11789/) (2021)
- [CRISP: Curriculum based Sequential Neural Decoders for Polar Code Family](https://mlsanity.com/paper/2210.00313/) (2022)
- [Neural Decoding with Optimization of Node Activations](https://mlsanity.com/paper/2206.00786/) (2022)
- [Feedback is Good, Active Feedback is Better: Block Attention Active Feedback Codes](https://mlsanity.com/paper/2211.01730/) (2022)
- [Robust Non-Linear Feedback Coding via Power-Constrained Deep Learning](https://mlsanity.com/paper/2304.13178/) (2023)
- [Generalization Bounds for Neural Belief Propagation Decoders](https://mlsanity.com/paper/2305.10540/) (2023)
- [LightCode: Light Analytical and Neural Codes for Channels with Feedback](https://mlsanity.com/paper/2403.10751/) (2024)
- [CMDNet: Learning a Probabilistic Relaxation of Discrete Variables for Soft Detection with Low Complexity](https://mlsanity.com/paper/2102.12756/) (2021)
- [Factor Graph Optimization of Error-Correcting Codes for Belief Propagation Decoding](https://mlsanity.com/paper/2406.12900/) (2024)
- [DeepIC: Coding for Interference Channels via Deep Learning](https://mlsanity.com/paper/2108.06028/) (2021)
- [Hybrid Neural Coded Modulation: Design and Training Methods](https://mlsanity.com/paper/2202.01972/) (2022)
- [Nested Construction of Polar Codes via Transformers](https://mlsanity.com/paper/2401.17188/) (2024)
- [DEEP-IoT: Downlink-Enhanced Efficient-Power Internet of Things](https://mlsanity.com/paper/2403.00321/) (2024)
- [Doubly Residual Neural Decoder: Towards Low-Complexity High-Performance Channel Decoding](https://mlsanity.com/paper/2102.03959/) (2021)
- [Improving the List Decoding Version of the Cyclically Equivariant Neural Decoder](https://mlsanity.com/paper/2106.07964/) (2021)
- [Discriminative Mutual Information Estimators for Channel Capacity Learning](https://mlsanity.com/paper/2107.03084/) (2021)
- [Machine Learning-Aided Efficient Decoding of Reed-Muller Subcodes](https://mlsanity.com/paper/2301.06251/) (2023)
- [ProductAE: Toward Deep Learning Driven Error-Correction Codes of Large Dimensions](https://mlsanity.com/paper/2303.16424/) (2023)

## Abstract (arXiv)

The design of codes for communicating reliably over a statistically well defined channel is an important endeavor involving deep mathematical research and wide-ranging practical applications. In this work, we present the first family of codes obtained via deep learning, which significantly beats state-of-the-art codes designed over several decades of research. The communication channel under consideration is the Gaussian noise channel with feedback, whose study was initiated by Shannon; feedback is known theoretically to improve reliability of communication, but no practical codes that do so have ever been successfully constructed. We break this logjam by integrating information theoretic insights harmoniously with recurrent-neural-network based encoders and decoders to create novel codes that outperform known codes by 3 orders of magnitude in reliability. We also demonstrate several desirable properties of the codes: (a) generalization to larger block lengths, (b) composability with known codes, (c) adaptation to practical constraints. This result also has broader ramifications for coding theory: even when the channel has a clear mathematical model, deep learning methodologies, when combined with channel-specific information-theoretic insights, can potentially beat state-of-the-art codes constructed over decades of mathematical research.