mlsanityData to 14 Oct 2026

Enhancing Robustness of Offline Reinforcement Learning Under Data Corruption via Sharpness-Aware Minimization

arXiv 2511.17568 ↗First posted 14 Nov 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline reinforcement learning (RL) is vulnerable to real-world data corruption, with even robust algorithms failing under challenging observation and mixture corruptions. We posit this failure stems from data corruption creating sharp minima in the loss landscape, leading to poor generalization. To address this, we are the first to apply Sharpness-Aware Minimization (SAM) as a general-purpose, plug-and-play optimizer for offline RL. SAM seeks flatter minima, guiding models to more robust parameter regions. We integrate SAM into strong baselines for data corruption: IQL, a top-performing offline RL algorithm in this setting, and RIQL, an algorithm designed specifically for data-corruption robustness. We evaluate them on D4RL benchmarks with both random and adversarial corruption. Our SAM-enhanced methods consistently and significantly outperform the original baselines. Visualizations of the reward surface confirm that SAM finds smoother solutions, providing strong evidence for its effectiveness in improving the robustness of offline RL agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 12 references that are in the map. First posted on arXiv 14 Nov 2025.

Builds on

10 of 12 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions1 Nov
  2. 2024Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling5 Jul
  3. 2023Corruption-Robust Offline Reinforcement Learning with General Function Approximation23 Oct
  4. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruption19 Oct
  5. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 3

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  3. 2022How Does Sharpness-Aware Minimization Minimize Sharpness?cited by 65

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Offline Reinforcement Learning with Corruption Robustness
  2. 2024Corruption Robust Offline Reinforcement Learning with Human Feedback
  3. 2021Corruption-Robust Offline Reinforcement Learning
  4. 2025Robust Policy Expansion for Offline-to-Online RL under Diverse Data Corruption
  5. 2025Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
  6. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothing
  7. 2025ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning
  8. 2023Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage