# Markov Persuasion Processes: Learning to Persuade from Scratch

arXiv: https://arxiv.org/abs/2402.03077 · HTML: https://mlsanity.com/paper/2402.03077/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 28 of 28)

- [Online Learning: A Modern Introduction Using Convex Optimization](https://mlsanity.com/paper/1912.13213/) (2019)
- [Exploration-Exploitation in Constrained MDPs](https://mlsanity.com/paper/2003.02189/) (2020)
- [Learning Adversarial MDPs with Bandit Feedback and Unknown Transition](https://mlsanity.com/paper/1912.01192/) (2019)
- [Online Convex Optimization in Adversarial Markov Decision Processes](https://mlsanity.com/paper/1905.07773/) (2019)
- [Constrained Upper Confidence Reinforcement Learning](https://mlsanity.com/paper/2001.09377/) (2020)
- [Upper Confidence Primal-Dual Reinforcement Learning for CMDP with Adversarial Loss](https://mlsanity.com/paper/2003.00660/) (2020)
- [Bayesian Exploration: Incentivizing Exploration in Bayesian Games](https://mlsanity.com/paper/1602.07570/) (2016)
- [Algorithmic Bayesian Persuasion](https://mlsanity.com/paper/1503.05988/) (2015)
- [Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning](https://mlsanity.com/paper/2202.10678/) (2022)
- [Optimal Rates and Efficient Algorithms for Online Bayesian Persuasion](https://mlsanity.com/paper/2303.01296/) (2023)
- [Online Learning in Weakly Coupled Markov Decision Processes: A Convergence Time Study](https://mlsanity.com/paper/1709.03465/) (2017)
- [Learning to Persuade on the Fly: Robustness Against Ignorance](https://mlsanity.com/paper/2102.10156/) (2021)
- [Bayesian Persuasion in Sequential Decision-Making](https://mlsanity.com/paper/2106.05137/) (2021)
- [Multi-Receiver Online Bayesian Persuasion](https://mlsanity.com/paper/2106.06480/) (2021)
- [Sequential Information Design: Learning to Persuade in the Dark](https://mlsanity.com/paper/2209.03927/) (2022)
- [A Best-of-Both-Worlds Algorithm for Constrained MDPs with Long-Term Constraints](https://mlsanity.com/paper/2304.14326/) (2023)
- [Information Design in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.06807/) (2023)
- [Targeting and Signaling in Ad Auctions](https://mlsanity.com/paper/1708.00611/) (2017)
- [Mixture Selection, Mechanism Design, and Signaling](https://mlsanity.com/paper/1508.03679/) (2015)
- [Hardness Results for Signaling in Bayesian Zero-Sum and Network Routing Games](https://mlsanity.com/paper/1512.03543/) (2015)
- [Signaling in Bayesian Network Congestion Games: the Subtle Power of Symmetry](https://mlsanity.com/paper/2002.05190/) (2020)
- [Persuading Voters: It's Easy to Whisper, It's Hard to Speak Loud](https://mlsanity.com/paper/1908.10620/) (2019)
- [Persuading Farsighted Receivers in MDPs: the Power of Honesty](https://mlsanity.com/paper/2306.12221/) (2023)
- [Persuading Voters in District-based Elections](https://mlsanity.com/paper/2012.05002/) (2020)
- [Public Signaling in Bayesian Ad Auctions](https://mlsanity.com/paper/2201.09728/) (2022)
- [Signaling in Posted Price Auctions](https://mlsanity.com/paper/2201.12183/) (2022)
- [Online Mechanism Design for Information Acquisition](https://mlsanity.com/paper/2302.02873/) (2023)
- [Markov Persuasion Processes with Endogenous Agent Beliefs](https://mlsanity.com/paper/2307.03181/) (2023)

## Built on by (most cited first, 8 of 8)

- [Optimal Strong Regret and Violation in Constrained MDPs via Policy Optimization](https://mlsanity.com/paper/2410.02275/) (2024)
- [Generalized Principal-Agent Problem with a Learning Agent](https://mlsanity.com/paper/2402.09721/) (2024)
- [Learning Constrained Markov Decision Processes With Non-stationary Rewards and Constraints](https://mlsanity.com/paper/2405.14372/) (2024)
- [Information Design with Unknown Prior](https://mlsanity.com/paper/2410.05533/) (2024)
- [A Primal-Dual Online Learning Approach for Dynamic Pricing of Sequentially Displayed Complementary Items under Sale Constraints](https://mlsanity.com/paper/2407.05793/) (2024)
- [A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models](https://mlsanity.com/paper/2504.04976/) (2025)
- [Provably Efficient Algorithm for Best Scoring Rule Identification in Online Principal-Agent Information Acquisition](https://mlsanity.com/paper/2505.17379/) (2025)
- [Meta-Learning for Repeated Bayesian Persuasion](https://mlsanity.com/paper/2603.20408/) (2026)

## Abstract (arXiv)

In Bayesian persuasion, an informed sender strategically discloses information to a receiver so as to persuade them to undertake desirable actions. Recently, a growing attention has been devoted to settings in which sender and receivers interact sequentially. Recently, Markov persuasion processes (MPPs) have been introduced to capture sequential scenarios where a sender faces a stream of myopic receivers in a Markovian environment. The MPPs studied so far in the literature suffer from issues that prevent them from being fully operational in practice, e.g., they assume that the sender knows receivers' rewards. We fix such issues by addressing MPPs where the sender has no knowledge about the environment. We design a learning algorithm for the sender, working with partial feedback. We prove that its regret with respect to an optimal information-disclosure policy grows sublinearly in the number of episodes, as it is the case for the loss in persuasiveness cumulated while learning. Moreover, we provide a lower bound for our setting matching the guarantees of our algorithm.