mlsanityData to 7 Oct 2026

Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning

arXiv 2209.08947 ↗First posted 19 Sep 2022cs.AI cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

The age of information metric fails to correctly describe the intrinsic semantics of a status update. In an intelligent reflecting surface-aided cooperative relay communication system, we propose the age of semantics (AoS) for measuring semantics freshness of the status updates. Specifically, we focus on the status updating from a source node (SN) to the destination, which is formulated as a Markov decision process (MDP). The objective of the SN is to maximize the expected satisfaction of AoS and energy consumption under the maximum transmit power constraint. To seek the optimal control policy, we first derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. However, implementing the online DAC in practice poses the key challenge in infinitely repeated interactions between the SN and the system, which can be dangerous particularly during the exploration. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset without any further interactions with the system. Numerical experiments verify the theoretical results and show that our offline DAC scheme significantly outperforms the online DAC scheme and the most representative baselines in terms of mean utility, demonstrating strong robustness to dataset quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 33 of its 73 references that are in the map. First posted on arXiv 19 Sep 2022.

Builds on

33 of 73 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  9. 2018Maximum a Posteriori Policy Optimisationcited by 318
  10. 2020Critic Regularized Regressioncited by 269
  11. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  12. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
21 more
  1. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  2. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  3. 2020Age of Information: An Introduction and Surveycited by 64
  4. 2021Offline Reinforcement Learning as Anti-Explorationcited by 44
  5. 2018Variational Bayesian Reinforcement Learning with Regret Boundscited by 31
  6. 2021Task-Oriented Image Transmission for Scene Classification in Unmanned Aerial Systemscited by 29
  7. 2018Optimized Computation Offloading Performance in Virtual Edge Computing Systems via Deep Reinforcement Learningcited by 26
  8. 2021Learning Stochastic Shortest Path with Linear Function Approximationcited by 23
  9. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 21
  10. 2018Multi-Tenant Cross-Slice Resource Orchestration: A Deep Reinforcement Learning Approachcited by 10
  11. 2021Distributed Reinforcement Learning for Age of Information Minimization in Real-Time IoT Systemscited by 10
  12. 2019Age of Information-Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspectivecited by 9
  13. 2019A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systemscited by 9
  14. 2019Ranking Policy Gradientcited by 4
  15. 2020ConQUR: Mitigating Delusional Bias in Deep Q-learningcited by 4
  16. 2020Information Freshness-Aware Task Offloading in Air-Ground Integrated Edge Computing Systemscited by 4
  17. 2021Information Freshness in Multi-Hop Wireless Networkscited by 3
  18. 2020Optimal Scheduling of Age-centric Caching: Tractability and Computationcited by 1
  19. 2020Optimizing Information Freshness in Two-Hop Status Update Systems under a Resource Constraintcited by 1
  20. 2021IRS-Aided Wireless Relaying: Optimal Deployment and Capacity Scalingcited by 1
  21. 2022Towards Industry 5.0: Intelligent Reflecting Surface (IRS) in Smart Manufacturingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Model-based Reinforcement Learning for Version Age of Information Scheduling: Average and Tail-Risk-Sensitive Control
  2. 2026Latency-aware Human-in-the-Loop Reinforcement Learning for Semantic Communications
  3. 2026Communications-Incentivized Collaborative Reasoning in NetGPT through Agentic Reinforcement Learning
  4. 2025Agentic Link Construction for Environment and Intent Aware 6G Communication
  5. 2024Knowledge Distillation from Language-Oriented to Emergent Communication for Multi-Agent Remote Control
  6. 2023Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach
  7. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs
  8. 2026LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learning