mlsanityData to 7 Oct 2026

Target-Aligned Fusion for Decision-Sequence Learning under Dynamics Shift

arXiv 2511.09173 ↗First posted 12 Nov 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

External trajectories can improve offline decision-sequence learning, but dynamics shift may make some source subsequences inconsistent with the target environment. We study how to fuse such trajectories with limited target data for Decision Transformer learning under dynamics shift. We propose Target-Aligned Fusion (TAF), a principled framework that derives source-data fusion from a target-domain Bellman-risk criterion. Our analysis bounds this risk by two measurable data-alignment quantities: Deltam, the state-structure mismatch of retained fragments, and Deltaw, the weighted transport cost from source to target transitions. This decomposition yields a gate--then--weight rule: source fragments are first filtered by target-side state-structure alignment, and retained transitions are then reweighted by local target feasibility. We instantiate this principle as TAF-DT, which uses maximum mean discrepancy (MMD) for fragment selection, optimal transport for feasibility-aware weighting, and the resulting fused law for advantage-token relabeling and Q-regularized Transformer training. Across gravity, kinematic, and morphology shifts on D4RL-style control tasks, TAF-DT achieves the strongest aggregate performance against strong offline RL and sequence-model baselines and produces more stable stitch-junction sequence semantics. Overall, these results indicate that aligning external trajectories to target-domain structure and feasibility is a practical way to exploit source data under dynamics shift.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 23 references that are in the map. First posted on arXiv 12 Nov 2025.

Builds on

20 of 23 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learning30 Oct
  2. 2024Q-value Regularized Transformer for Offline Reinforcement Learning27 May
  3. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning21 Dec
  4. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning12 Sep
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 15

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2016OpenAI Gymcited by 1,684
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  8. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  9. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  10. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
  11. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitchingcited by 35
  12. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learningcited by 27
3 more
  1. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  2. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  3. 2025DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learningcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Hybrid Sequence Modeling and Reinforced Verification for Controllable Target-Conditioned Decision Making
  2. 2026Target-Aligned Bellman Backup for Cross-domain Offline Reinforcement Learning
  3. 2026Trust Guided Decision Transformer
  4. 2026Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
  5. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  6. 2023Multi-Objective Decision Transformers for Offline Reinforcement Learning
  7. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targets
  8. 2024Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation