mlsanityData to 7 Oct 2026

Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning

arXiv 2512.08485 ↗First posted 9 Dec 2025cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

also in 1 more
Abstract (arXiv)

Offline Reinforcement Learning (RL) enables policy optimization from static datasets but is inherently vulnerable to data poisoning attacks. Existing attack strategies typically rely on locally uniform perturbations, which treat all samples indiscriminately. This approach is inefficient, as it wastes the perturbation budget on low-impact samples, and lacks stealthiness due to significant statistical deviations. In this paper, we propose a novel Global Budget Allocation attack strategy. Leveraging the theoretical insight that a sample's influence on value function convergence is proportional to its Temporal Difference (TD) error, we formulate the attack as a global resource allocation problem. We derive a closed-form solution where perturbation magnitudes are assigned proportional to the TD-error sensitivity under a global L2 constraint. Empirical results on D4RL benchmarks demonstrate that our method significantly outperforms baseline strategies, achieving up to 80% performance degradation with minimal perturbations that evade detection by state-of-the-art statistical and spectral defenses.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 16 references that are in the map. First posted on arXiv 9 Dec 2025.

Builds on

15 of 16 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Multi-level Certified Defense Against Poisoning Attacks in Offline Reinforcement Learning27 May
  2. 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets7 Oct
  3. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks16 Mar
  4. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  7. 2019Policy Poisoning in Batch Reinforcement Learning and Control13 Oct

Other references · 8

  1. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  3. 2023Mastering Diverse Domains through World Modelscited by 1,193
  4. 2017Adversarial Attacks on Neural Network Policiescited by 314
  5. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  6. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  7. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  8. 2019TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agentscited by 29

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning
  2. 2026CS-GBA: A Critical Sample-based Gradient-guided Backdoor Attack for Offline Reinforcement Learning
  3. 2022Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
  4. 2023Data Poisoning to Fake a Nash Equilibrium in Markov Games
  5. 2026Sequential Data Poisoning in LLM Post-Training
  6. 2024Universal Black-Box Reward Poisoning Attack against Offline Reinforcement Learning
  7. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
  8. 2023Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources