# Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signals

arXiv: https://arxiv.org/abs/1906.10571 · HTML: https://mlsanity.com/paper/1906.10571/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 30 of 50)

- [Robust Reinforcement Learning on State Observations with Learned Optimal Adversary](https://mlsanity.com/paper/2101.08452/) (2021)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL](https://mlsanity.com/paper/2106.05087/) (2021)
- [Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning](https://mlsanity.com/paper/2210.05927/) (2022)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning](https://mlsanity.com/paper/2110.04471/) (2021)
- [Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments](https://mlsanity.com/paper/2102.08492/) (2021)
- [COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks](https://mlsanity.com/paper/2203.08398/) (2022)
- [Defense Against Reward Poisoning Attacks in Reinforcement Learning](https://mlsanity.com/paper/2102.05776/) (2021)
- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2206.01888/) (2022)
- [Reinforcement Learning for Feedback-Enabled Cyber Resilience](https://mlsanity.com/paper/2107.00783/) (2021)
- [Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence](https://mlsanity.com/paper/2302.03322/) (2023)
- [Deceptive Reinforcement Learning for Privacy-Preserving Planning](https://mlsanity.com/paper/2102.03022/) (2021)
- [Multi-Task Federated Reinforcement Learning with Adversaries](https://mlsanity.com/paper/2103.06473/) (2021)
- [Admissible Policy Teaching through Reward Design](https://mlsanity.com/paper/2201.02185/) (2022)
- [Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks](https://mlsanity.com/paper/2207.10170/) (2022)
- [Efficient Adversarial Attacks on Online Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2307.07670/) (2023)
- [Game Redesign in No-regret Game Playing](https://mlsanity.com/paper/2110.11763/) (2021)
- [Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies](https://mlsanity.com/paper/2402.12673/) (2024)
- [New Challenges in Reinforcement Learning: A Survey of Security and Privacy](https://mlsanity.com/paper/2301.00188/) (2022)
- [Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysis](https://mlsanity.com/paper/2208.00081/) (2022)
- [Reward Delay Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/2209.03540/) (2022)
- [Corruption-Robust Offline Two-Player Zero-Sum Markov Games](https://mlsanity.com/paper/2403.07933/) (2024)
- [Improving Robustness of Deep Reinforcement Learning Agents: Environment Attack based on the Critic Network](https://mlsanity.com/paper/2104.03154/) (2021)
- [Reward-Free Attacks in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2112.00940/) (2021)
- [Efficient Action Poisoning Attacks on Linear Contextual Bandits](https://mlsanity.com/paper/2112.05367/) (2021)
- [Minimally Modifying a Markov Game to Achieve Any Nash Equilibrium and Value](https://mlsanity.com/paper/2311.00582/) (2023)
- [Security of Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2212.06123/) (2022)
- [Efficient Action Robust Reinforcement Learning with Probabilistic Policy Execution Uncertainty](https://mlsanity.com/paper/2307.07666/) (2023)

## Abstract (arXiv)

This paper studies reinforcement learning (RL) under malicious falsification on cost signals and introduces a quantitative framework of attack models to understand the vulnerabilities of RL. Focusing on Q-learning, we show that Q-learning algorithms converge under stealthy attacks and bounded falsifications on cost signals. We characterize the relation between the falsified cost and the Q-factors as well as the policy learned by the learning agent which provides fundamental limits for feasible offensive and defensive moves. We propose a robust region in terms of the cost within which the adversary can never achieve the targeted policy. We provide conditions on the falsified cost which can mislead the agent to learn an adversary's favored policy. A numerical case study of water reservoir control is provided to show the potential hazards of RL in learning-based control systems and corroborate the results.