# Joint Learning of Probabilistic and Geometric Shaping for Coded Modulation Systems

arXiv: https://arxiv.org/abs/2004.05062 · HTML: https://mlsanity.com/paper/2004.05062/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2020.

## Built on by (most cited first, 5 of 5)

- [DeepJSCC-Q: Constellation Constrained Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2206.08100/) (2022)
- [Neural Capacity Estimators: How Reliable Are They?](https://mlsanity.com/paper/2111.07401/) (2021)
- [End-to-End Learning of Joint Geometric and Probabilistic Constellation Shaping](https://mlsanity.com/paper/2112.05050/) (2021)
- [Neural Probabilistic Shaping: Joint Distribution Learning for Optical Fiber Communications](https://mlsanity.com/paper/2507.16012/) (2025)
- [Sequential Neural Probabilistic Amplitude Shaping: Learning the Channel's Language](https://mlsanity.com/paper/2605.28143/) (2026)

## Abstract (arXiv)

We introduce a trainable coded modulation scheme that enables joint optimization of the bit-wise mutual information (BMI) through probabilistic shaping, geometric shaping, bit labeling, and demapping for a specific channel model and for a wide range of signal-to-noise ratios (SNRs). Compared to probabilistic amplitude shaping (PAS), the proposed approach is not restricted to symmetric probability distributions, can be optimized for any channel model, and works with any code rate k/m, m being the number of bits per channel use and k an integer within the range from 1 to m-1. The proposed scheme enables learning of a continuum of constellation geometries and probability distributions determined by the SNR. Additionally, the PAS architecture with Maxwell-Boltzmann (MB) as shaping distribution was extended with a neural network (NN) that controls the MB shaping of a quadrature amplitude modulation (QAM) constellation according to the SNR, enabling learning of a continuum of MB distributions for QAM. Simulations were performed to benchmark the performance of the proposed joint probabilistic and geometric shaping scheme on additive white Gaussian noise (AWGN) and mismatched Rayleigh block fading (RBF) channels.