# Utility Maximizing Sequential Sensing Over a Finite Horizon

arXiv: https://arxiv.org/abs/1705.05960 · HTML: https://mlsanity.com/paper/1705.05960/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2017.

## Built on by (most cited first, 1 of 1)

- [Learning-based Spectrum Sensing and Access in Cognitive Radios via Approximate POMDPs](https://mlsanity.com/paper/2107.07049/) (2021)

## Abstract (arXiv)

We consider the problem of optimally utilizing N resources, each in an unknown binary state. The state of each resource can be inferred from state-dependent noisy measurements. Depending on its state, utilizing a resource results in either a reward or a penalty per unit time. The objective is a sequential strategy governing the decision of sensing and exploitation at each time to maximize the expected utility (i.e., total reward minus total penalty and sensing cost) over a finite horizon L. We formulate the problem as a Partially Observable Markov Decision Process (POMDP) and show that the optimal strategy is based on two time-varying thresholds for each resource and an optimal selection rule for which resource to sense. Since a full characterization of the optimal strategy is generally intractable, we develop a low-complexity policy that is shown by simulations to offer near optimal performance. This problem finds applications in opportunistic spectrum access, marketing strategies and other sequential resource allocation problems.