mlsanityData to 14 Oct 2026

Retentive Decision Transformer with Adaptive Masking for Reinforcement Learning based Recommendation Systems

arXiv 2403.17634 ↗First posted 26 Mar 2024cs.IR cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

also in 1 more
Abstract (arXiv)

Reinforcement Learning-based Recommender Systems (RLRS) have shown promise across a spectrum of applications, from e-commerce platforms to streaming services. Yet, they grapple with challenges, notably in crafting reward functions and harnessing large pre-existing datasets within the RL framework. Recent advancements in offline RLRS provide a solution for how to address these two challenges. However, existing methods mainly rely on the transformer architecture, which, as sequence lengths increase, can introduce challenges associated with computational resources and training costs. Additionally, the prevalent methods employ fixed-length input trajectories, restricting their capacity to capture evolving user preferences. In this study, we introduce a new offline RLRS method to deal with the above problems. We reinterpret the RLRS challenge by modeling sequential decision-making as an inference task, leveraging adaptive masking configurations. This adaptive approach selectively masks input tokens, transforming the recommendation task into an inference challenge based on varying token subsets, thereby enhancing the agent's ability to infer across diverse trajectory lengths. Furthermore, we incorporate a multi-scale segmented retention mechanism that facilitates efficient modeling of long sequences, significantly enhancing computational efficiency. Our experimental analysis, conducted on both online simulator and offline datasets, clearly demonstrates the advantages of our proposed method.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 44 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

27 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Causal Decision Transformer for Recommender Systems via Offline Reinforcement Learning17 Apr
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 24

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2018Group Normalizationcited by 1,524
  5. 2018Self-Attentive Sequential Recommendationcited by 1,184
  6. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  7. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  8. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  9. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  10. 2021Reinforcement learning based recommender systems: A surveycited by 188
  11. 2019Behavior Sequence Transformer for E-commerce Recommendation in Alibabacited by 130
  12. 2019Reinforcement Knowledge Graph Reasoning for Explainable Recommendationcited by 102
12 more
  1. 2018Deep Reinforcement Learning for Page-wise Recommendationscited by 100
  2. 2018Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learningcited by 80
  3. 2018Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learningcited by 71
  4. 2023Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendationcited by 38
  5. 2023Automated Self-Supervised Learning for Recommendationcited by 37
  6. 2019Model-Based Reinforcement Learning with Adversarial Training for Online Recommendationcited by 33
  7. 2018Large-scale Interactive Recommendation with Tree-structured Policy Gradientcited by 30
  8. 2023KuaiSim: A Comprehensive Simulator for Recommender Systemscited by 30
  9. 2020Knowledge-guided Deep Reinforcement Learning for Interactive Recommendationcited by 18
  10. 2023User Retention-oriented Recommendation with Decision Transformercited by 14
  11. 2020Generative Inverse Deep Reinforcement Learning for Online Recommendationcited by 12
  12. 2023Transformers in Reinforcement Learning: A Surveycited by 12

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Energy-Guided Diffusion Sampling for Long-Term User Behavior Prediction in Reinforcement Learning-based Recommendationcited by 1
  2. 2025Adaptive Learning Systems: Personalized Curriculum Design Using LLM-Powered Analyticscited by 0
  3. 2025Generative Sequential Notification Optimization via Multi-Objective Decision Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Robust Reinforcement Learning Objectives for Sequential Recommender Systems
  2. 2024ROLeR: Effective Reward Shaping in Offline Reinforcement Learning for Recommender Systems
  3. 2025DARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward
  4. 2023Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation
  5. 2021RL4RS: A Real-World Dataset for Reinforcement Learning based Recommender System
  6. 2023On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems
  7. 2020Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation
  8. 2024EasyRL4Rec: An Easy-to-use Library for Reinforcement Learning Based Recommender Systems