# Interpreting Training Aspects of Deep-Learned Error-Correcting Codes

arXiv: https://arxiv.org/abs/2305.04347 · HTML: https://mlsanity.com/paper/2305.04347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 7 May 2023.

## Builds on (most cited first, 10 of 10)

- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [The staircase property: How hierarchical structure can guide deep learning](https://mlsanity.com/paper/2108.10573/) (2021)
- [Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels](https://mlsanity.com/paper/1911.03038/) (2019)
- [Deepcode: Feedback Codes via Deep Learning](https://mlsanity.com/paper/1807.00801/) (2018)
- [KO codes: Inventing Nonlinear Encoding and Decoding for Reliable Wireless Communication via Deep-learning](https://mlsanity.com/paper/2108.12920/) (2021)
- [Learning to Communicate: Channel Auto-encoders, Domain Specific Regularizers, and Attention](https://mlsanity.com/paper/1608.06409/) (2016)
- [Neural Distributed Source Coding](https://mlsanity.com/paper/2106.02797/) (2021)
- [MIND: Model Independent Neural Decoder](https://mlsanity.com/paper/1903.02268/) (2019)
- [All you need is feedback: Communication with block attention feedback codes](https://mlsanity.com/paper/2206.09457/) (2022)
- [LEARN Codes: Inventing Low-latency Codes via Recurrent Neural Networks](https://mlsanity.com/paper/1811.12707/) (2018)

## Abstract (arXiv)

As new deep-learned error-correcting codes continue to be introduced, it is important to develop tools to interpret the designed codes and understand the training process. Prior work focusing on the deep-learned TurboAE has both interpreted the learned encoders post-hoc by mapping these onto nearby ``interpretable'' encoders, and experimentally evaluated the performance of these interpretable encoders with various decoders. Here we look at developing tools for interpreting the training process for deep-learned error-correcting codes, focusing on: 1) using the Goldreich-Levin algorithm to quickly interpret the learned encoder; 2) using Fourier coefficients as a tool for understanding the training dynamics and the loss landscape; 3) reformulating the training loss, the binary cross entropy, by relating it to encoder and decoder parameters, and the bit error rate (BER); 4) using these insights to formulate and study a new training procedure. All tools are demonstrated on TurboAE, but are applicable to other deep-learned forward error correcting codes (without feedback).