Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentation
On the map: Offline RL and imitation learning from data → Offline RL algorithms
Abstract (arXiv)
Current offline reinforcement learning (ORL) algorithms tend to overfit the training dataset and exhibit poor in-distribution generalization and robustness performance when deployed to real environments, thus compromising their effectiveness. Existing methods typically enhance in-distribution generalization and robustness by leveraging regularization techniques widely used in computer vision. However, due to the high sensitivity of low-level physical signals to distributional shifts, these methods still suffer from notable limitations in in-distribution generalization and robustness, making it difficult to achieve stable performance in complex environments. To address this issue, we theoretically analyze the error bounds of the behavior policy and action-value function trained with random episode interpolation, revealing that the error scales positively correlated with the distance between states. Based on this insight, we propose a method called bfBoundary-bfAware bfData bfAugmentation (BADA), which leverages neighboring states to construct interpolation boundaries, enabling the generation of synthetic data that more faithfully preserves the original data distribution. We first conduct qualitative studies in a toy environment, showing that BADA generates mixed samples that preserve desirable policy smoothness while accurately reconstructing multimodal value distributions. Extensive experiments on limited offline datasets further demonstrate that BADA attains state-of-the-art performance across diverse benchmarks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 46 references that are in the map. First posted on arXiv 22 Aug 2026.
Builds on
36 of 46 references.
Closest ancestors · 14
Same direction, newest first
- 2025Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning4 Nov
- 2025Offline Reinforcement Learning with Penalized Action Noise Injection3 Jul
- 2025Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity20 Jun
- 2025Model-Based Offline Reinforcement Learning with Adversarial Data Augmentation26 Mar
- 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions1 Nov
- 2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning14 Mar
- 2023The Generalization Gap in Offline Reinforcement Learning10 Dec
- 2023MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator7 Dec
- 2023Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL6 Oct
- 2023Uncertainty-driven Trajectory Truncation for Data Augmentation in Offline Reinforcement Learning10 Apr
- 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
- 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
2 more
Other references · 22
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2016Adversarial examples in the physical worldcited by 1,769
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
- 2020Reinforcement Learning with Augmented Datacited by 468
- 2022G-Mixup: Graph Data Augmentation for Graph Classificationcited by 146
- 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
- 2024Normalization and effective learning rates in reinforcement learningcited by 58
- 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 27
10 more
- 2020Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Modelscited by 26
- 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25
- 2024Robust Reinforcement Learning from Corrupted Human Feedbackcited by 25
- 2023The Benefits of Mixup for Feature Learningcited by 19
- 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learningcited by 15
- 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
- 2021A Survey on Deep Reinforcement Learning for Data Processing and Analyticscited by 10
- 2024A Survey Analyzing Generalization in Deep Reinforcement Learningcited by 8
- 2023Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imaginationcited by 6
- 2025Reinforcement Learning Platform for Adversarial Black-box Attacks with Custom Distortion Filterscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Synthetic Data is Sufficient for Zero-Shot Visual Generalization from Offline Data
- 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
- 2026Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL
- 2022Domain Generalization for Robust Model-Based Offline Reinforcement Learning
- 2024Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning
- 2024Doubly Mild Generalization for Offline Reinforcement Learning
- 2024Integrating Domain Knowledge for handling Limited Data in Offline RL
- 2023Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations