# Joint Age-State Belief is All You Need: Minimizing AoII via Pull-Based Remote Estimation

arXiv: https://arxiv.org/abs/2411.07179 · HTML: https://mlsanity.com/paper/2411.07179/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Partially Observable Minimum-Age Scheduling: The Greedy Policy](https://mlsanity.com/paper/2009.13441/) (2020)
- [Indexability and Rollout Policy for Multi-State Partially Observable Restless Bandits](https://mlsanity.com/paper/2108.00892/) (2021)
- [Age-of-Information-based Scheduling in Multiuser Uplinks with Stochastic Arrivals: A POMDP Approach](https://mlsanity.com/paper/2005.05443/) (2020)
- [Status Updating under Partial Battery Knowledge in Energy Harvesting IoT Networks](https://mlsanity.com/paper/2303.18104/) (2023)
- [Query-Based Sampling of Heterogeneous CTMCs: Modeling and Optimization with Binary Freshness](https://mlsanity.com/paper/2310.02223/) (2023)
- [Collaborative Optimization of the Age of Information under Partial Observability](https://mlsanity.com/paper/2312.12977/) (2023)
- [Modeling AoII in Push- and Pull-Based Sampling of Continuous Time Markov Chains](https://mlsanity.com/paper/2401.04098/) (2024)
- [AoII-Optimum Sampling of CTMC Information Sources Under Sampling Rate Constraints](https://mlsanity.com/paper/2401.18063/) (2024)
- [Multi-Threshold AoII-Optimum Sampling Policies for CTMC Information Sources](https://mlsanity.com/paper/2407.08592/) (2024)

## Abstract (arXiv)

Age of incorrect information (AoII) is a recently proposed freshness and mismatch metric that penalizes an incorrect estimation along with its duration. Therefore, keeping track of AoII requires the knowledge of both the source and estimation processes. In this paper, we consider a time-slotted pull-based remote estimation system under a sampling rate constraint where the information source is a general discrete-time Markov chain (DTMC) process. Moreover, packet transmission times from the source to the monitor are non-zero which disallows the monitor to have perfect information on the actual AoII process at any time. Hence, for this pull-based system, we propose the monitor to maintain a sufficient statistic called em belief which stands for the joint distribution of the age and source processes to be obtained from the history of all observations. Using belief, we first propose a maximum a posteriori (MAP) estimator to be used at the monitor as opposed to existing martingale estimators in the literature. Second, we obtain the optimality equations from the belief-MDP (Markov decision process) formulation. Finally, we propose two belief-dependent policies one of which is based on deep reinforcement learning, and the other one is a threshold-based policy based on the instantaneous expected AoII.