mlsanityData to 14 Oct 2026

Reinforcement Learning for Jointly Optimal Coding and Control Policies for a Controlled Markovian System over a Communication Channel

arXiv 2411.13884 ↗First posted 21 Nov 2024math.OCcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We study the problem of joint optimization involving coding and control policies for a controlled Markovian sytem over a finite-rate noiseless communication channel. While structural results on the optimal encoding and control have been obtained in the literature, their implementation has been prohibitive in general, except for linear models. We develop regularity and existence results on optimal policies. We then obtain rigorous approximation and near optimality results for jointly optimal coding and control policies. To this end, we first develop existence, regularity, and structural properties on optimal policies, followed by rigorous approximations and reinforcement learning results. Notably, we establish near optimality of finite model approximations obtained via predictor quantization as well as sliding finite window approximations, and their reinforcement learning convergence to near optimality. A detailed comparison of the approximation schemes and their reinforcement learning performance is presented.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 73 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

11 of 73 references.

References · 11

  1. 2020Near Optimality of Finite Memory Feedback Policies in Partially Observed Markov Decision Processescited by 23
  2. 2015Asymptotic Optimality of Finite Approximations to Markov Decision Processes with Borel Spacescited by 14
  3. 2021Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuitycited by 11
  4. 2016Rate-cost tradeoffs in controlcited by 9
  5. 2015LQG Control with Minimum Directed Information: Semidefinite Programming Approachcited by 8
  6. 2023Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environmentscited by 7
  7. 2014Semidefinite Programming Approach to Gaussian Sequential Rate-Distortion Trade-offscited by 4
  8. 2018Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sourcescited by 3
  9. 2023Reinforcement Learning for Near-Optimal Design of Zero-Delay Codes for Markov Sourcescited by 3
  10. 2022Time-invariant Prefix Coding for LQG Controlcited by 2
  11. 2014Separated design of encoder and controller for networked linear quadratic optimal controlcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025On Policy Stochasticity in Mutual Information Optimal Control of Linear Systems
  2. 2024Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning
  3. 2026Principled Learning-to-Communicate with Quasi-Classical Information Structures
  4. 2024On Convergence of Average-Reward Q-Learning in Weakly Communicating Markov Decision Processes
  5. 2025Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis
  6. 2026Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication
  7. 2024A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms
  8. 2024Near Optimal Approximations and Finite Memory Policies for POMPDs with Continuous Spaces ↗