# Collaborative Optimization of the Age of Information under Partial Observability

arXiv: https://arxiv.org/abs/2312.12977 · HTML: https://mlsanity.com/paper/2312.12977/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Age of Information: An Introduction and Survey](https://mlsanity.com/paper/2007.08564/) (2020)
- [Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learning](https://mlsanity.com/paper/1910.12802/) (2019)
- [Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysis](https://mlsanity.com/paper/2002.04131/) (2020)
- [Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games](https://mlsanity.com/paper/2212.14449/) (2022)
- [On the Role of Age of Information in the Internet of Things](https://mlsanity.com/paper/1812.08286/) (2018)
- [Age of Information-Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspective](https://mlsanity.com/paper/1908.02047/) (2019)
- [A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQ](https://mlsanity.com/paper/2102.09774/) (2021)
- [Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behavior](https://mlsanity.com/paper/2307.06175/) (2023)
- [Partially Observable Minimum-Age Scheduling: The Greedy Policy](https://mlsanity.com/paper/2009.13441/) (2020)
- [Dynamic Pricing and Mean Field Analysis for Controlling Age of Information](https://mlsanity.com/paper/2004.10050/) (2020)
- [Throughput and Age of Information in a Cellular-based IoT Network](https://mlsanity.com/paper/2005.09547/) (2020)
- [Age of Information in Ultra-Dense IoT Systems: Performance and Mean-Field Game Analysis](https://mlsanity.com/paper/2006.15756/) (2020)
- [A Palm Calculus Approach to the Distribution of the Age of Information](https://mlsanity.com/paper/2204.04643/) (2022)
- [Statistical Age-of-Information Bounds for Parallel Systems: When Do Independent Channels Make a Difference?](https://mlsanity.com/paper/2303.14035/) (2023)

## Built on by (most cited first, 1 of 1)

- [Joint Age-State Belief is All You Need: Minimizing AoII via Pull-Based Remote Estimation](https://mlsanity.com/paper/2411.07179/) (2024)

## Abstract (arXiv)

The significance of the freshness of sensor and control data at the receiver side, often referred to as Age of Information (AoI), is fundamentally constrained by contention for limited network resources. Evidently, network congestion is detrimental for AoI, where this congestion is partly self-induced by the sensor transmission process in addition to the contention from other transmitting sensors. In this work, we devise a decentralized AoI-minimizing transmission policy for a number of sensor agents sharing capacity-limited, non-FIFO duplex channels that introduce random delays in communication with a common receiver. By implementing the same policy, however with no explicit inter-agent communication, the agents minimize the expected AoI in this partially observable system. We cater to the partial observability due to random channel delays by designing a bootstrap particle filter that independently maintains a belief over the AoI of each agent. We also leverage mean-field control approximations and reinforcement learning to derive scalable and optimal solutions for minimizing the expected AoI collaboratively.