# Deceptive Sequential Decision-Making via Regularized Policy Optimization

arXiv: https://arxiv.org/abs/2501.18803 · HTML: https://mlsanity.com/paper/2501.18803/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 41 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 16 of 16)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward](https://mlsanity.com/paper/1905.12762/) (2019)
- [Manipulation Attacks in Local Differential Privacy](https://mlsanity.com/paper/1909.09630/) (2019)
- [Entropy Maximization for Markov Decision Processes Under Temporal Logic Constraints](https://mlsanity.com/paper/1807.03223/) (2018)
- [Are We There Yet? Timing and Floating-Point Attacks on Differential Privacy Systems](https://mlsanity.com/paper/2112.05307/) (2021)
- [Privacy-Preserving Policy Synthesis in Markov Decision Processes](https://mlsanity.com/paper/2004.07778/) (2020)
- [A Note on Optimization Formulations of Markov Decision Processes](https://mlsanity.com/paper/2012.09417/) (2020)
- [Deception in Supervisory Control](https://mlsanity.com/paper/1902.00590/) (2019)
- [Differential Privacy for Symbolic Systems with Application to Markov Chains](https://mlsanity.com/paper/2202.03325/) (2022)
- [Least Inferable Policies for Markov Decision Processes](https://mlsanity.com/paper/1809.06482/) (2018)
- [Differentially Private LQ Control](https://mlsanity.com/paper/1807.05082/) (2018)
- [Differential Privacy in Cooperative Multiagent Planning](https://mlsanity.com/paper/2301.08811/) (2023)
- [Guaranteed Feasibility in Differentially Private Linearly Constrained Convex Optimization](https://mlsanity.com/paper/2409.08364/) (2024)
- [Markov Decision Process to Enforce Moving Target Defence Policies](https://mlsanity.com/paper/1905.09222/) (2019)
- [Differentially Private Formation Control: Privacy and Network Co-Design](https://mlsanity.com/paper/2205.13406/) (2022)

## Built on by (most cited first, 1 of 1)

- [Value of Information-based Deceptive Path Planning Under Adversarial Interventions](https://mlsanity.com/paper/2503.24284/) (2025)

## Abstract (arXiv)

Autonomous systems are increasingly expected to operate in the presence of adversaries, though adversaries may infer sensitive information simply by observing a system. Therefore, present a deceptive sequential decision-making framework that not only conceals sensitive information, but actively misleads adversaries about it. We model autonomous systems as Markov decision processes, with adversaries using inverse reinforcement learning to recover reward functions. To counter them, we present three regularization strategies for policy synthesis problems that actively deceive an adversary about a system's reward. ``Diversionary deception'' leads an adversary to draw any false conclusion about the system's reward function. ``Targeted deception'' leads an adversary to draw a specific false conclusion about the system's reward function. ``Equivocal deception'' leads an adversary to infer that the real reward and a false reward both explain the system's behavior. We show how each form of deception can be implemented in policy optimization problems and analytically bound the loss in total accumulated reward induced by deception. Next, we evaluate these developments in a multi-agent setting. We show that diversionary, targeted, and equivocal deception all steer the adversary to false beliefs while still attaining a total accumulated reward that is at least 98% of its optimal, non-deceptive value.