# Structured Reinforcement Learning for Bayesian Persuasion : Application to Intelligent Interactive Driving

arXiv: https://arxiv.org/abs/2607.13576 · HTML: https://mlsanity.com/paper/2607.13576/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)
- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 15 of 15)

- [Bayesian Exploration: Incentivizing Exploration in Bayesian Games](https://mlsanity.com/paper/1602.07570/) (2016)
- [Algorithmic Bayesian Persuasion](https://mlsanity.com/paper/1503.05988/) (2015)
- [Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment](https://mlsanity.com/paper/1909.11538/) (2019)
- [Bayesian Persuasion in Sequential Decision-Making](https://mlsanity.com/paper/2106.05137/) (2021)
- [Sequential Information Design: Learning to Persuade in the Dark](https://mlsanity.com/paper/2209.03927/) (2022)
- [Information Design in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.06807/) (2023)
- [Persuading a Behavioral Agent: Approximately Best Responding and Learning](https://mlsanity.com/paper/2302.03719/) (2023)
- [Bayesian Persuasive Driving](https://mlsanity.com/paper/1809.09735/) (2018)
- [Multiple Stopping Time POMDPs: Structural Results & Application in Interactive Advertising in Social Media](https://mlsanity.com/paper/1712.00524/) (2017)
- [Optimal Scheduling of Multiple Sensors over Lossy and Bandwidth Limited Channels](https://mlsanity.com/paper/1804.05618/) (2018)
- [Persuading Farsighted Receivers in MDPs: the Power of Honesty](https://mlsanity.com/paper/2306.12221/) (2023)
- [Decision-Oriented Learning with Differentiable Submodular Maximization for Vehicle Routing Problem](https://mlsanity.com/paper/2303.01543/) (2023)
- [Approximation Algorithms for Inventory Problems with Submodular or Routing Costs](https://mlsanity.com/paper/1504.06560/) (2015)
- [A Framework for Efficient Dynamic Routing under Stochastically Varying Conditions](https://mlsanity.com/paper/2111.11095/) (2021)
- [A Novel Method for Lane-change Maneuver in Urban Driving Using Predictive Markov Decision Process](https://mlsanity.com/paper/2212.12008/) (2022)

## Abstract (arXiv)

Interactive driving, wherein an intelligent lead vehicle equipped with real-time traffic data coordinates route choices of connected vehicles, offers a promising approach to dynamic traffic management. To address the challenge of harmonising decisions, this paper considers the strategic information revealing framework of Bayesian persuasion. Here, the principal (lead vehicle) aims to guide the agent's (connected vehicle) partially observable sequential decision making towards its own objectives by selectively revealing information, such as real-time traffic ahead, using signals. However, the agent's farsighted response to maximize its long-term reward, renders the principal's signaling strategy design computationally challenging. We propose an online structured reinforcement learning framework to synthesize computationally efficient signaling strategy which is persuasive for a far-sighted agent. The main contributions of the paper are as follows: (i) For a monotonic agent with approximate best response, we propose MAPL, a structured policy learning algorithm for faster online learning, (ii) Identification of sufficient conditions for the supermodular structure of the Q function of the principal for a monotonic agent, (iii) Identification of sufficient conditions to ensure the persuasiveness of the principal's signaling strategy, (iv) Supermodular Q learning for Principal (SQP), which leverages the supermodular structure of principal's action value to synthesize computationally efficient signaling strategy that is persuasive for a monotonic learning agent, (v) Numerical analysis considering a real-time application of Bayesian persuasive driving for lane selection demonstrates that the proposed method is 30% cost efficient for optimising travelling rewards of both the lead and connected vehicle compared to the existing methodologies for signaling strategy design.