mlsanityData to 14 Oct 2026

Trajectory-Level Data Augmentation for Offline Reinforcement Learning

arXiv 2605.13401 ↗First posted 13 May 2026cs.LG cs.RO stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We propose a data augmentation method for offline reinforcement learning, motivated by active positioning problems. Particularly, our approach enables the training of off-policy models from a limited number of suboptimal trajectories. We introduce a trajectory-based augmentation technique that exploits task structure and the geometric relationship between rewards, value functions, and mathematical properties of logging policies. During data collection, our augmentation supports suboptimal logging policies, leading to higher data quality and improved offline reinforcement learning performance. We provide theoretical justification for these strategies and validate them empirically across positioning tasks of varying dimensionality and under partial observability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 39 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

29 of 39 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning27 May
  2. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching4 Feb
  3. 2023Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning27 Oct
  4. 2023Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting22 Jun
  5. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learning16 May
  6. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?12 Apr
  7. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning31 Jan
  8. 2021A Dataset Perspective on Offline Reinforcement Learning8 Nov
  9. 2021d3rlpy: An Offline Deep Reinforcement Learning Library6 Nov
  10. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  11. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  12. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning10 Mar
2 more
  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 15

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  5. 2020Reinforcement Learning with Augmented Datacited by 468
  6. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  7. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
  8. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  9. 2023Synthetic Experience Replaycited by 101
  10. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  11. 2017Markov Decision Processes with Continuous Side Informationcited by 56
  12. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 27
3 more
  1. 2020Interferobot: aligning an optical interferometer by a reinforcement learning agentcited by 12
  2. 2025Behavioral Exploration: Learning to Explore via In-Context Adaptationcited by 5
  3. 2025Active Alignments of Lens Systems with Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Augmenting Offline Reinforcement Learning with State-only Interactions
  2. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learning
  3. 2022Model-based Trajectory Stitching for Improved Offline Reinforcement Learning
  4. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environments
  5. 2025BiTrajDiff: Bidirectional Trajectory Generation with Diffusion Models for Offline Reinforcement Learning
  6. 2025Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning
  7. 2023Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation
  8. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets