mlsanityData to 14 Oct 2026

CoDrift: Compositional Drifting for Offline Reinforcement Learning

arXiv 2608.23939 ↗First posted 25 Aug 2026cs.LG cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline reinforcement learning is intrinsically multi-objective: a policy must remain compatible with the behavioral support of a fixed dataset while preferentially selecting high-value actions. We recast these objectives in a common form by viewing each as an action-space motion field that specifies how generated actions should move. This perspective enables heterogeneous learning objectives to be combined directly through field composition. Inspired by drifting models, we propose CoDrift, a compositional framework for one-step generative policy learning. CoDrift combines three objective-level fields into a unified policy field. The conditional field preserves state-dependent behavioral structure, while the marginal field pools actions across states to provide a more stable generative signal in the single-positive-sample regime of continuous-control offline RL. The value field moves generated actions toward higher-value regions. The composed field is absorbed into a stochastic generator that produces an action with a single forward pass at deployment. We evaluate CoDrift on 73 tasks from OGBench and D4RL in both offline and offline-to-online settings. CoDrift compares favorably with state-of-the-art methods and achieves the best average rank in both settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 33 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

31 of 33 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Scaling Offline RL via Efficient and Expressive Shortcut Models28 May
  2. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learning16 May
  3. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization28 Mar
  4. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning9 Mar
  5. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  6. 2021Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matters8 Oct
  7. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  8. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  9. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  10. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  11. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  12. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 19

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  5. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  6. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  7. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  8. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  9. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 325
  10. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  11. 2025Flow Q-Learningcited by 184
  12. 2026Generative Modeling via Driftingcited by 116
7 more
  1. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 100
  2. 2023Score Regularized Policy Optimization through Diffusion Behaviorcited by 71
  3. 2025One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlowcited by 18
  4. 2026DeFlow: Decoupling Manifold Modeling and Value Maximization for Offline Policy Extractioncited by 5
  5. 2025EfficientFlow: Efficient Equivariant Flow Policy Learning for Embodied AIcited by 4
  6. 2026Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flowcited by 3
  7. 2026BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning
  2. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
  3. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism
  4. 2025MOBODY: Model Based Off-Dynamics Offline Reinforcement Learning
  5. 2025Offline Reinforcement Learning with Generative Trajectory Policies
  6. 2024An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces
  7. 2023Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
  8. 2025Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model