# Model-Based Deep Learning of Joint Probabilistic and Geometric Shaping for Optical Communication

arXiv: https://arxiv.org/abs/2204.07457 · HTML: https://mlsanity.com/paper/2204.07457/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Introduction to Deep Learning for the Physical Layer](https://mlsanity.com/paper/1702.00832/) (2017)
- [Joint Learning of Geometric and Probabilistic Constellation Shaping](https://mlsanity.com/paper/1906.07748/) (2019)
- [End-to-end Learning for GMI Optimized Geometric Constellation Shape](https://mlsanity.com/paper/1907.08535/) (2019)
- [Deep-learning Autoencoder for Coherent and Nonlinear Optical Communication](https://mlsanity.com/paper/2006.15027/) (2020)
- [End-to-end Autoencoder for Superchannel Transceivers with Hardware Impairment](https://mlsanity.com/paper/2103.15856/) (2021)
- [Increasing Achievable Information Rates via Geometric Shaping](https://mlsanity.com/paper/1804.08850/) (2018)

## Built on by (most cited first, 2 of 2)

- [Geometric Constellation Shaping for Fiber-Optic Channels via End-to-End Learning](https://mlsanity.com/paper/2211.04311/) (2022)
- [Neural Probabilistic Shaping: Joint Distribution Learning for Optical Fiber Communications](https://mlsanity.com/paper/2507.16012/) (2025)

## Abstract (arXiv)

Autoencoder-based deep learning is applied to jointly optimize geometric and probabilistic constellation shaping for optical coherent communication. The optimized constellation shaping outperforms the 256 QAM Maxwell-Boltzmann probabilistic distribution with extra 0.05 bits/4D-symbol mutual information for 64 GBd transmission over 170 km SMF link.