# Semantic Rate Distortion and Posterior Design: Compute Constraints, Multimodality, and Strategic Inference

arXiv: https://arxiv.org/abs/2602.03949 · HTML: https://mlsanity.com/paper/2602.03949/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws](https://mlsanity.com/paper/2404.05405/) (2024)
- [Low-Rank Bottleneck in Multi-head Attention Models](https://mlsanity.com/paper/2002.07028/) (2020)
- [How much do language models memorize?](https://mlsanity.com/paper/2505.24832/) (2025)
- [Algorithmic Bayesian Persuasion](https://mlsanity.com/paper/1503.05988/) (2015)
- [Rate-Distortion Theory for Strategic Semantic Communication](https://mlsanity.com/paper/2202.03711/) (2022)
- [L2M: Mutual Information Scaling Law for Long-Context Language Modeling](https://mlsanity.com/paper/2503.04725/) (2025)
- [A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Search](https://mlsanity.com/paper/2507.00004/) (2025)
- [Dynamic Information Design: A Simple Problem on Optimal Sequential Information Disclosure](https://mlsanity.com/paper/2003.07965/) (2020)
- [Degenerate Kalman filter error covariances and their convergence onto the unstable subspace](https://mlsanity.com/paper/1604.02578/) (2016)
- [Persuasion with Limited Communication](https://mlsanity.com/paper/1606.07843/) (2016)
- [Information-Theoretic Approach to Strategic Communication as a Hierarchical Game](https://mlsanity.com/paper/1510.00764/) (2015)
- [Persuasion with limited communication capacity](https://mlsanity.com/paper/1711.04474/) (2017)
- [Persuasion with Coarse Communication](https://mlsanity.com/paper/1910.13547/) (2019)
- [Information Design for Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2207.06411/) (2022)
- [LQG Information Design](https://mlsanity.com/paper/2312.09479/) (2023)

## Abstract (arXiv)

We study strategic Gaussian semantic compression under rate and compute constraints, where an encoder and decoder optimize distinct quadratic objectives. A latent Gaussian state generates a task dependent semantic variable, and the decoder best responds via MMSE estimation, reducing the encoder's problem to posterior covariance design under an information rate constraint. We characterize the strategic rate distortion function in direct, remote, and full information regimes, derive semantic waterfilling and rate constrained Gaussian persuasion solutions, and establish Gaussian optimality under misaligned objectives. We further show that architectural compute limits act as implicit rate constraints, yielding exponential improvements in semantic accuracy with model depth and inference time compute, while multimodal observation eliminates the geometric mean penalty inherent to remote encoding. These results provide information theoretic foundations for data and energy efficient AI and offer a principled interpretation of modern multimodal language models as posterior design mechanisms under resource constraints.