mlsanityData to 7 Oct 2026

Semantic Rate-Distortion for Bounded Multi-Agent Communication: Capacity-Derived Semantic Spaces and the Communication Cost of Alignment

arXiv 2604.09521 ↗First posted 10 Apr 2026cs.IT cs.AI math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

When two agents of different computational capacities interact with the same environment, they need not compress a common semantic alphabet differently; they can induce different semantic alphabets altogether. We show that the quotient POMDP Qm,T(M) - the unique coarsest abstraction consistent with an agent's capacity - serves as a capacity-derived semantic space for any bounded agent, and that communication between heterogeneous agents exhibits a sharp structural phase transition. Below a critical rate Rcrit determined by the quotient mismatch, intent-preserving communication is structurally impossible. In the supported one-way memoryless regime, classical side-information coding then yields exponential decay above the induced benchmark. Classical coding theorems tell you the rate once the source alphabet is fixed; our contribution is to derive that alphabet from bounded interaction itself. Concretely, we prove: (1) a fixed-varepsilon structural phase-transition theorem whose lower bound is fully general on the common-history quotient comparison; (2) a one-way Wyner-Ziv benchmark identification on quotient alphabets, with exact converse, exact operational equality for memoryless quotient sources, and an ergodic long-run bridge via explicit mixing bounds; (3) an asymptotic one-way converse in the shrinking-distortion regime varepsilon = O(1/T), proved from the message stream and decoder side information; and (4) alignment traversal bounds enabling compositional communication through intermediate capacity levels. Experiments on eight POMDP environments (including RockSample(4,4)) illustrate the phase transition, a structured-policy benchmark shows the one-way rate can drop by up to 19× relative to the counting bound, and a shrinking-distortion sweep matches the regime of the asymptotic converse.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 50 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

14 of 50 references.

References · 14

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  6. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  7. 2020Deep Learning Enabled Semantic Communication Systemscited by 282
  8. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  9. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  10. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 166
  11. 2020Semantics-Empowered Communication for Networked Intelligent Systemscited by 60
  12. 2019Biases for Emergent Communication in Multi-agent Reinforcement Learningcited by 48
2 more
  1. 2020On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Viewscited by 28
  2. 2026The Myhill-Nerode Theorem for Bounded Interaction: Canonical Abstractions via Agent-Bounded Indistinguishabilitycited by 2

Built on by

Not cited yet.