# Nested Construction of Polar Codes via Transformers

arXiv: https://arxiv.org/abs/2401.17188 · HTML: https://mlsanity.com/paper/2401.17188/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Online Decision Transformer](https://mlsanity.com/paper/2202.05607/) (2022)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [Deep Learning Methods for Improved Decoding of Linear Codes](https://mlsanity.com/paper/1706.07043/) (2017)
- [Learning to Decode Linear Codes Using Deep Learning](https://mlsanity.com/paper/1607.04793/) (2016)
- [Error Correction Code Transformer](https://mlsanity.com/paper/2203.14966/) (2022)
- [Communication Algorithms via Deep Learning](https://mlsanity.com/paper/1805.09317/) (2018)
- [Deepcode: Feedback Codes via Deep Learning](https://mlsanity.com/paper/1807.00801/) (2018)
- [ViterbiNet: A Deep Learning Based Viterbi Algorithm for Symbol Detection](https://mlsanity.com/paper/1905.10750/) (2019)
- [Massive MIMO Channel Estimation with an Untrained Deep Neural Network](https://mlsanity.com/paper/1908.00144/) (2019)
- [KO codes: Inventing Nonlinear Encoding and Decoding for Reliable Wireless Communication via Deep-learning](https://mlsanity.com/paper/2108.12920/) (2021)
- [ProductAE: Towards Training Larger Channel Codes based on Neural Product Codes](https://mlsanity.com/paper/2110.04466/) (2021)
- [Deep Learning-based Polar Code Design](https://mlsanity.com/paper/1909.12035/) (2019)
- [Construction of Polar Codes with Reinforcement Learning](https://mlsanity.com/paper/2009.09277/) (2020)
- [CRISP: Curriculum based Sequential Neural Decoders for Polar Code Family](https://mlsanity.com/paper/2210.00313/) (2022)
- [Decoder-tailored Polar Code Design Using the Genetic Algorithm](https://mlsanity.com/paper/1901.10464/) (2019)
- [Interpreting Neural Min-Sum Decoders](https://mlsanity.com/paper/2205.10684/) (2022)
- [TinyTurbo: Efficient Turbo Decoders on Edge](https://mlsanity.com/paper/2209.15614/) (2022)
- [Data-Driven Neural Polar Codes for Unknown Channels With and Without Memory](https://mlsanity.com/paper/2309.03148/) (2023)
- [Reinforcement Learning for Nested Polar Code Construction](https://mlsanity.com/paper/1904.07511/) (2019)
- [Deep Polar Codes](https://mlsanity.com/paper/2308.03004/) (2023)

## Built on by (most cited first, 2 of 2)

- [LightCode: Light Analytical and Neural Codes for Channels with Feedback](https://mlsanity.com/paper/2403.10751/) (2024)
- [DeepPolar: Inventing Nonlinear Large-Kernel Polar Codes via Deep Learning](https://mlsanity.com/paper/2402.08864/) (2024)

## Abstract (arXiv)

Tailoring polar code construction for decoding algorithms beyond successive cancellation has remained a topic of significant interest in the field. However, despite the inherent nested structure of polar codes, the use of sequence models in polar code construction is understudied. In this work, we propose using a sequence modeling framework to iteratively construct a polar code for any given length and rate under various channel conditions. Simulations show that polar codes designed via sequential modeling using transformers outperform both 5G-NR sequence and Density Evolution based approaches for both AWGN and Rayleigh fading channels.