Embedding Information in Policy Randomness: Distribution-Preserving Implicit Communication in MDPs
Not on the map. Why?
Abstract (arXiv)
This paper investigates a new role of randomness in stochastic policies--as a medium for embedding and communicating information. Specifically, we consider an agent interacting with an environment according to a prescribed stochastic policy. By carefully modulating the random action realizations of the policy, information can be embedded into the agent's behavior without altering the prescribed action distribution. We cast this problem as a channel coding problem, in which the controlled system implicitly serves as a communication channel. We then propose a coding scheme, termed posterior-matching randomness modulation (PRAM), that selects actions according to the message to be transmitted. With the aid of common randomness independent of the control system, PRAM preserves the prescribed action distribution for every state and every message. We further prove that PRAM is asymptotically reliable: the message can be decoded with arbitrarily small error probability as the observed state trajectory becomes sufficiently long. Experiments demonstrate the effectiveness of the proposed framework in applications including multi-bit policy watermarking and trajectory provenance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 11 Oct 2026.
Builds on
20 of 33 references.
References · 20
- 2023A Watermark for Large Language Modelscited by 725
- 2018Understanding the impact of entropy on policy optimizationcited by 176
- 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
- 2019Deep Reinforcement Learning for Cyber Securitycited by 95
- 2023Advancing Beyond Identification: Multi-bit Watermark for Large Language Modelscited by 57
- 2023Towards Codable Watermarking for Injecting Multi-bits Information to LLMscited by 54
- 2019Maximum Entropy-Regularized Multi-Goal Reinforcement Learningcited by 40
- 2020A Review of Deep Reinforcement Learning for Smart Building Energy Managementcited by 38
- 2021Communicating via Markov Decision Processescited by 10
- 2025Agent Guide: A Simple Agent Behavioral Watermarking Frameworkcited by 10
- 2019Sequential Triggers for Watermarking of Deep Reinforcement Learning Policiescited by 8
- 2025WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarkingcited by 6
8 more
- 2026Sequential Behavioral Watermarking for LLM Agentscited by 5
- 2026TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectoriescited by 5
- 2018Least Inferable Policies for Markov Decision Processescited by 4
- 2026QuantileMark: A Message-Symmetric Multi-bit Watermark for LLMscited by 3
- 2024Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Controlcited by 2
- 2025Remotely Detectable Robot Policy Watermarkingcited by 2
- 2025Communication over LQG Control Systems: A Convex Optimization Approach to Capacitycited by 1
- 2026The Control Plant as A Communication Channel: Implicit Communication for Decentralized LQG Controlcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Principled Learning-to-Communicate with Quasi-Classical Information Structures
- 2024Information-Theoretic Opacity-Enforcement in Markov Decision Processes
- 2025Policy Gradient Methods for Information-Theoretic Opacity in Markov Decision Processes
- 2026Decoupling Communication from Policy: Robust MARL under Bandwidth Constraints
- 2025Synthesis of Dynamic Masks for Information-Theoretic Opacity in Stochastic Systems
- 2025Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning
- 2024Reinforcement Learning for Jointly Optimal Coding and Control Policies for a Controlled Markovian System over a Communication Channel
- 2026CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination ↗