# Rethinking GSPO: The Perplexity-Entropy Equivalence

arXiv: https://arxiv.org/abs/2510.23142 · HTML: https://mlsanity.com/paper/2510.23142/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 16 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Builds on (most cited first, 4 of 4)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)

## Built on by (most cited first, 1 of 1)

- [Rewarding Structural Conformance of Reasoning using Process Mining](https://mlsanity.com/paper/2510.25065/) (2025)

## Abstract (arXiv)

We provide a new perspective on GSPO's length-normalized importance ratios by establishing their connection to information-theoretic quantities. We show that GSPO's sequence-level weight s(θ) = (πθ/π_θold)1/|y| can be equivalently expressed as the inverse perplexity ratio PPL_θold/PPLθ and as the exponential cross-entropy change exp(Delta H). While the perplexity-entropy relationship follows from standard definitions, this observation provides a useful lens for understanding GSPO: the algorithm weights policy gradient updates by perplexity ratios, offering an information-theoretic interpretation of the importance weights. This perspective helps explain GSPO's empirical properties, including log-domain variance reduction through geometric averaging and stability in training mixture-of-experts models. We validate the mathematical equivalences and variance predictions through controlled experiments on mathematical reasoning tasks.