mlsanityData to 7 Oct 2026

Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning

arXiv 2602.21072 ↗First posted 24 Feb 2026cs.LG cs.AI cs.ROcited by 1

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically address dynamics mismatch either globally over the state space or via pointwise data filtering; these approaches can miss localized cross-domain similarities or incur high computational cost. We propose Localized Dynamics-Aware Domain Adaptation (LoDADA), which exploits localized dynamics mismatch to better reuse source data. LoDADA clusters transitions from source and target datasets and estimates cluster-level dynamics discrepancy via domain discrimination. Source transitions from clusters with small discrepancy are retained, while those from clusters with large discrepancy are filtered out. This yields a fine-grained and scalable data selection strategy that avoids overly coarse global assumptions and expensive per-sample filtering. We provide theoretical insights and extensive experiments across environments with diverse global and local dynamics shifts. Results show that LoDADA consistently outperforms state-of-the-art off-dynamics offline RL methods by better leveraging localized distribution mismatch.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 35 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

29 of 35 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025MOBODY: Model Based Off-Dynamics Offline Reinforcement Learning10 Jun
  2. 2024Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learning10 May
  3. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning22 Jun
  4. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learning13 Mar
  5. 2021Offline Reinforcement Learning with Soft Behavior Regularization14 Oct
  6. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  7. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation21 Jun
  8. 2020MOPO: Model-based Offline Policy Optimization27 May
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  10. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 19

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  3. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  4. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  5. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  6. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 138
  7. 2018Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithmcited by 111
  8. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  9. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 93
  10. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  11. 2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
  12. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
7 more
  1. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  2. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimizationcited by 22
  3. 2024Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learningcited by 15
  4. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 15
  5. 2024Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learningcited by 13
  6. 2025Sample Complexity of Distributionally Robust Off-Dynamics Reinforcement Learning with Online Interactioncited by 7
  7. 2025Policy Regularized Distributionally Robust Markov Decision Processes with Linear Function Approximationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correction
  2. 2025Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filtering
  3. 2024Provable Domain Adaptation for Offline Reinforcement Learning with Limited Samples
  4. 2021Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL
  5. 2022Domain Generalization for Robust Model-Based Offline Reinforcement Learning
  6. 2026Unifying Value Alignment and Assignment in Cross-Domain Offline Reinforcement Learning with Heterogeneous Datasets
  7. 2025Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learning
  8. 2024Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning