mlsanityData to 7 Oct 2026

Balancing detectability and performance of attacks on the control channel of Markov Decision Processes

arXiv 2109.07171 ↗First posted 15 Sep 2021eess.SY cs.CR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We investigate the problem of designing optimal stealthy poisoning attacks on the control channel of Markov decision processes (MDPs). This research is motivated by the recent interest of the research community for adversarial and poisoning attacks applied to MDPs, and reinforcement learning (RL) methods. The policies resulting from these methods have been shown to be vulnerable to attacks perturbing the observations of the decision-maker. In such an attack, drawing inspiration from adversarial examples used in supervised learning, the amplitude of the adversarial perturbation is limited according to some norm, with the hope that this constraint will make the attack imperceptible. However, such constraints do not grant any level of undetectability and do not take into account the dynamic nature of the underlying Markov process. In this paper, we propose a new attack formulation, based on information-theoretical quantities, that considers the objective of minimizing the detectability of the attack as well as the performance of the controlled process. We analyze the trade-off between the efficiency of the attack and its detectability. We conclude with examples and numerical simulations illustrating this trade-off.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 55 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

25 of 55 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2016Adversarial examples in the physical worldcited by 1,762
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  10. 2017Constrained Policy Optimizationcited by 928
  11. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  12. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
13 more
  1. 2017Robust Adversarial Reinforcement Learningcited by 439
  2. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
  3. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  4. 2017Adversarial Attacks on Neural Network Policiescited by 314
  5. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  6. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  7. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  8. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  9. 2020Adversarial Machine Learning -- Industry Perspectivescited by 121
  10. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  11. 2020Robustifying Reinforcement Learning Agents via Action Space Adversarial Trainingcited by 30
  12. 2019Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agentscited by 24
  13. 2021Minimizing Information Leakage of Abrupt Changes in Stochastic Systemscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 1.

4 papers.

  1. 2022Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attackscited by 12
  2. 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
  3. 2023Learning-Based Latency-Constrained Fronthaul Compression Optimization in C-RANcited by 1
  4. 2025Diffusion Guided Adversarial State Perturbations in Reinforcement Learningcited by 0