End-to-end optimization of coherent optical communications over the split-step Fourier method guided by the nonlinear Fourier transform theory

Optimizing modulation and detection strategies for a given channel is critical to maximize the throughput of a communication system. Such an optimization can be easily carried out analytically for channels that admit closed-form analytical models. However, this task becomes extremely challenging for nonlinear dispersive channels such as the optical fiber. End-to-end optimization through autoencoders (AEs) can be applied to define symbol-to-waveform (modulation) and waveform-to-symbol (detection) mappings, but so far it has been mainly shown for systems relying on approximate channel models. Here, for the first time, we propose an AE scheme applied to the full optical channel described by the nonlinear Schr"odinger equation (NLSE). Transmitter and receiver are jointly optimized through the split-step Fourier method (SSFM) which accurately models an optical fiber. In this first numerical analysis, the detection is performed by a neural network (NN), whereas the symbol-to-waveform mapping is aided by the nonlinear Fourier transform (NFT) theory in order to simplify and guide the optimization on the modulation side. This proof-of-concept AE scheme is thus benchmarked against a standard NFT-based system and a threefold increase in achievable distance (from 2000 to 6640 km) is demonstrated.