mlsanityData to 14 Oct 2026

Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce

arXiv 2512.13726 ↗First posted 13 Dec 2025cs.LG cs.AIcited by 3

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Unlike traditional recommendation tasks, finite user time budgets introduce a critical resource constraint, requiring the recommender system to balance item relevance and evaluation cost. For example, in a mobile shopping interface, users interact with recommendations by scrolling, where each scroll triggers a list of items called slate. Users incur an evaluation cost - time spent assessing item features before deciding to click. Highly relevant items having higher evaluation costs may not fit within the user's time budget, affecting engagement. In this position paper, our objective is to evaluate reinforcement learning algorithms that learn patterns in user preferences and time budgets simultaneously, crafting recommendations with higher engagement potential under resource constraints. Our experiments explore the use of reinforcement learning to recommend items for users using Alibaba's Personalized Re-ranking dataset supporting slate optimization in e-commerce contexts. Our contributions include (i) a unified formulation of time-constrained slate recommendation modeled as Markov Decision Processes (MDPs) with budget-aware utilities; (ii) a simulation framework to study policy behavior on re-ranking data; and (iii) empirical evidence that on-policy and off-policy control can improve performance under tight time budgets than traditional contextual bandit-based methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 26 references that are in the map. First posted on arXiv 13 Dec 2025.

Builds on

15 of 26 references.

References · 15

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2021Reinforcement learning based recommender systems: A surveycited by 188
  3. 2016Off-policy evaluation for slate recommendationcited by 103
  4. 2018Deep Reinforcement Learning for Page-wise Recommendationscited by 100
  5. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  6. 2017Deep Reinforcement Learning for List-wise Recommendationscited by 35
  7. 2018Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modelingcited by 31
  8. 2019Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodologycited by 27
  9. 2021A Survey on Reinforcement Learning for Recommender Systemscited by 18
  10. 2015Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actionscited by 13
  11. 2023Generative Slate Recommendation with Reinforcement Learningcited by 13
  12. 2024Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approachcited by 10
3 more
  1. 2024MM-PoE: Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Modelscited by 5
  2. 2023Distributional Off-Policy Evaluation for Slate Recommendationscited by 2
  3. 2025CLT and Edgeworth Expansion for m-out-of-n Bootstrap Estimators of The Studentized Mediancited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendationcited by 1
  2. 2026Is Sliding Window All You Need? An Open Framework for Long-Sequence Recommendationcited by 0
  3. 2026DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Tradingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems
  2. 2026FlexRec: Adapting LLM-based Recommenders for Flexible Needs via Reinforcement Learning
  3. 2024Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation
  4. 2026LLM-Enhanced Reinforcement Learning for Long-Term User Satisfaction in Interactive Recommendation
  5. 2025Time Matters: Enhancing Sequential Recommendations with Time-Guided Graph Neural ODEs ↗
  6. 2024UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systems
  7. 2025RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems
  8. 2020Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation