mlsanityData to 14 Oct 2026

TSN-Affinity: Similarity-Driven Parameter Reuse for Continual Offline Reinforcement Learning

arXiv 2604.25898 ↗First posted 28 Apr 2026cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Continual offline reinforcement learning (CORL) aims to learn a sequence of tasks from datasets collected over time while preserving performance on previously learned tasks. This setting corresponds to domains where new tasks arise over time, but adapting the model in live environment interactions is expensive, risky, or impossible. However, CORL inherits the dual difficulty of offline reinforcement learning and adapting while preventing catastrophic forgetting. Replay-based continual learning approaches remain a strong baseline but incur memory overhead and suffer from a distribution mismatch between replayed samples and newly learned policies. At the same time, architectural continual learning methods have shown strong potential in supervised learning but remain underexplored in CORL. In this work, we propose TSN-Affinity, a novel CORL method based on TinySubNetworks and Decision Transformer. The method enables task-specific parameterization and controlled knowledge sharing through a RL-aware reuse strategy that routes tasks according to action compatibility and latent similarity. We evaluate the approach on benchmarks based on Atari games and simulations of manipulation tasks with the Franka Emika Panda robotic arm, covering both discrete and continuous control. Results show strong retention from sparse SubNetworks, with routing further improving multi-task performance. Our findings suggest that similarity-guided architectural reuse is a strong and viable alternative to replay-based strategies in a CORL setting. Our code is available at: https://github.com/anonymized-for-submission123/tsn-affinity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 21 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

17 of 21 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Solving Continual Offline RL through Selective Weights Activation on Aligned Spaces21 Oct
  2. 2024P2DT: Mitigating Forgetting in task-incremental Learning with progressive prompt Decision Transformer22 Jan
  3. 2022Multi-Game Decision Transformers30 May
  4. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 12

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2018Experience Replay for Continual Learningcited by 1,095
  4. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  5. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  6. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  7. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  8. 2023A Definition of Continual Reinforcement Learningcited by 101
  9. 2023Learning to Modulate pre-trained Models in RLcited by 28
  10. 2023Replay-enhanced Continual Reinforcement Learningcited by 7
  11. 2023From MNIST to ImageNet and Back: Benchmarking Continual Curriculum Learningcited by 6
  12. 2024TinySubNets: An efficient and low capacity continual learning strategycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023OER: Offline Experience Replay for Continual Offline Reinforcement Learning
  2. 2024Continual Task Learning through Adaptive Policy Self-Composition
  3. 2024Solving Continual Offline Reinforcement Learning with Decision Transformer
  4. 2024Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning
  5. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replay
  6. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
  7. 2025A Survey of Continual Reinforcement Learning
  8. 2026How to Find and Reuse Policies for Continuous Adaptation in Lifelong Reinforcement Learning