# Beyond Freshness and Semantics: A Coupon-Collector Framework for Effective Status Updates

arXiv: https://arxiv.org/abs/2603.26998 · HTML: https://mlsanity.com/paper/2603.26998/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 11 of 11)

- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems](https://mlsanity.com/paper/1809.05149/) (2018)
- [Age-of-Information vs. Value-of-Information Scheduling for Cellular Networked Control Systems](https://mlsanity.com/paper/1903.05356/) (2019)
- [Sampling for Data Freshness Optimization: Non-linear Age Functions](https://mlsanity.com/paper/1812.07241/) (2018)
- [Urgency of Information for Context-Aware Timely Status Updates in Remote Control Systems](https://mlsanity.com/paper/2002.07987/) (2020)
- [Age of Loop for Wireless Networked Control Systems Optimization](https://mlsanity.com/paper/2106.00415/) (2021)
- [Age of Actuation in a Wireless Power Transfer System](https://mlsanity.com/paper/2303.00507/) (2023)
- [Bulk-Switching Memristor-based Compute-In-Memory Module for Deep Neural Network Training](https://mlsanity.com/paper/2305.14547/) (2023)
- [Age and Value of Information Optimization for Systems with Multi-Class Updates](https://mlsanity.com/paper/2408.12589/) (2024)

## Abstract (arXiv)

For status update systems operating over unreliable energy-constrained wireless channels, we address Weaver's long-standing Level-C question: do my packets actually improve the plant's behavior? Each fresh sample carries a stochastic expiration time -- governed by the plant's instability dynamics -- after which the information becomes useless for control. Casting the problem as a coupon-collector variant with expiring coupons, we (i) formulate a two-dimensional average-reward MDP, (ii) prove that the optimal schedule is doubly thresholded in the receiver's freshness timer and the sender's stored lifetime, (iii) derive a closed-form policy for deterministic lifetimes, and (iv) design a Structure-Aware Q-learning algorithm (SAQ) that learns the optimal policy without knowing the channel success probability or lifetime distribution. Simulations validate our theoretical predictions: SAQ matches optimal Value Iteration performance while converging significantly faster than baseline Q-learning, and expiration-aware scheduling achieves up to 50% higher reward than age-based baselines by adapting transmissions to state-dependent urgency -- thereby delivering Level-C effectiveness under tight resource constraints.