# Deep Learning for Communication over Dispersive Nonlinear Channels: Performance and Comparison with Classical Digital Signal Processing

arXiv: https://arxiv.org/abs/1910.01028 · HTML: https://mlsanity.com/paper/1910.01028/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 1 of 1)

- [End-to-End Deep Learning of Long-Haul Coherent Optical Fiber Communications via Regular Perturbation Model](https://mlsanity.com/paper/2107.12320/) (2021)

## Abstract (arXiv)

In this paper, we apply deep learning for communication over dispersive channels with power detection, as encountered in low-cost optical intensity modulation/direct detection (IM/DD) links. We consider an autoencoder based on the recently proposed sliding window bidirectional recurrent neural network (SBRNN) design to realize the transceiver for optical IM/DD communication. We show that its performance can be improved by introducing a weighted sequence estimation scheme at the receiver. Moreover, we perform bit-to-symbol mapping optimization to reduce the bit-error rate (BER) of the system. Furthermore, we carry out a detailed comparison with classical schemes based on pulse-amplitude modulation and maximum likelihood sequence detection (MLSD). Our investigation shows that for a reference 42,Gb/s transmission, the SBRNN autoencoder achieves a BER performance comparable to MLSD, when both systems account for the same amount of memory. In contrast to MLSD, the SBRNN performance is achieved without incurring a computational complexity exponentially growing with the processed memory.