# Piecewise-Stationary Multi-Objective Multi-Armed Bandit with Application to Joint Communications and Sensing

arXiv: https://arxiv.org/abs/2302.05257 · HTML: https://mlsanity.com/paper/2302.05257/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 18 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 6 of 6)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [A Change-Detection based Framework for Piecewise-stationary Multi-Armed Bandit Problem](https://mlsanity.com/paper/1711.03539/) (2017)
- [Multi-armed Bandits with Application to 5G Small Cells](https://mlsanity.com/paper/1510.00627/) (2015)
- [Multi-objective Contextual Multi-armed Bandit with a Dominant Objective](https://mlsanity.com/paper/1708.05655/) (2017)
- [A Non-Stationary Bandit-Learning Approach to Energy-Efficient Femto-Caching with Rateless-Coded Transmission](https://mlsanity.com/paper/2005.04154/) (2020)
- [UCBoost: A Boosting Approach to Tame Complexity and Optimality for Stochastic Bandits](https://mlsanity.com/paper/1804.05929/) (2018)

## Built on by (most cited first, 3 of 3)

- [Provably Efficient Multi-Objective Bandit Algorithms under Preference-Centric Customization](https://mlsanity.com/paper/2502.13457/) (2025)
- [Federated Learning in UAV-Enhanced Networks: Joint Coverage and Convergence Time Optimization](https://mlsanity.com/paper/2308.16889/) (2023)
- [Provably Efficient Personalized Multi-Objective Bandits with Proactive Conversational Queries](https://mlsanity.com/paper/2606.08410/) (2026)

## Abstract (arXiv)

We study a multi-objective multi-armed bandit problem in a dynamic environment. The problem portrays a decision-maker that sequentially selects an arm from a given set. If selected, each action produces a reward vector, where every element follows a piecewise-stationary Bernoulli distribution. The agent aims at choosing an arm among the Pareto optimal set of arms to minimize its regret. We propose a Pareto generic upper confidence bound (UCB)-based algorithm with change detection to solve this problem. By developing the essential inequalities for multi-dimensional spaces, we establish that our proposal guarantees a regret bound in the order of γTlog(T/γT) when the number of breakpoints γT is known. Without this assumption, the regret bound of our algorithm is γTlog(T). Finally, we formulate an energy-efficient waveform design problem in an integrated communication and sensing system as a toy example. Numerical experiments on the toy example and synthetic and real-world datasets demonstrate the efficiency of our policy compared to the current methods.