# Neural Turbo Equalization: Deep Learning for Fiber-Optic Nonlinearity Compensation

arXiv: https://arxiv.org/abs/1911.10131 · HTML: https://mlsanity.com/paper/1911.10131/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2019.

## Built on by (most cited first, 4 of 4)

- [Domain Adaptation: the Key Enabler of Neural Network Equalizers in Coherent Optical Systems](https://mlsanity.com/paper/2202.12689/) (2022)
- [Hybrid Neural Coded Modulation: Design and Training Methods](https://mlsanity.com/paper/2202.01972/) (2022)
- [Turbo-ICL: In-Context Learning-Based Turbo Equalization](https://mlsanity.com/paper/2505.06175/) (2025)
- [Component Training of Turbo Autoencoders](https://mlsanity.com/paper/2305.09216/) (2023)

## Abstract (arXiv)

Recently, data-driven approaches motivated by modern deep learning have been applied to optical communications in place of traditional model-based counterparts. The application of deep neural networks (DNN) allows flexible statistical analysis of complicated fiber-optic systems without relying on any specific physical models. Due to the inherent nonlinearity in DNN, various equalizers based on DNN have shown significant potentials to mitigate fiber nonlinearity. In this paper, we propose a turbo equalization (TEQ) based on DNN as a new alternative framework to deal with nonlinear fiber impairments for future coherent optical communications. The proposed DNN-TEQ is constructed with nested deep residual networks (ResNet) to train extrinsic likelihood given soft-information feedback from channel decoding. Through extrinsic information transfer (EXIT) analysis, we verify that our DNN-TEQ can accelerate decoding convergence to achieve a significant gain in achievable throughput by 0.61b/s/Hz. We also demonstrate that optimizing irregular low-density parity-check (LDPC) codes to match EXIT chart of the DNN-TEQ can improve achievable rates by up to 0.12 b/s/Hz.