# Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce

arXiv: https://arxiv.org/abs/2512.13726 · HTML: https://mlsanity.com/paper/2512.13726/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 13 Dec 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 15 of 15)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Reinforcement learning based recommender systems: A survey](https://mlsanity.com/paper/2101.06286/) (2021)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Deep Reinforcement Learning for Page-wise Recommendations](https://mlsanity.com/paper/1805.02343/) (2018)
- [Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems](https://mlsanity.com/paper/1902.05570/) (2019)
- [Deep Reinforcement Learning for List-wise Recommendations](https://mlsanity.com/paper/1801.00209/) (2017)
- [Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling](https://mlsanity.com/paper/1810.12027/) (2018)
- [Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodology](https://mlsanity.com/paper/1905.12767/) (2019)
- [A Survey on Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2109.10665/) (2021)
- [Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actions](https://mlsanity.com/paper/1512.01124/) (2015)
- [Generative Slate Recommendation with Reinforcement Learning](https://mlsanity.com/paper/2301.08632/) (2023)
- [Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approach](https://mlsanity.com/paper/2412.00726/) (2024)
- [MM-PoE: Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Models](https://mlsanity.com/paper/2412.07148/) (2024)
- [Distributional Off-Policy Evaluation for Slate Recommendations](https://mlsanity.com/paper/2308.14165/) (2023)
- [CLT and Edgeworth Expansion for m-out-of-n Bootstrap Estimators of The Studentized Median](https://mlsanity.com/paper/2505.11725/) (2025)

## Built on by (most cited first, 3 of 3)

- [PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendation](https://mlsanity.com/paper/2601.06458/) (2026)
- [Is Sliding Window All You Need? An Open Framework for Long-Sequence Recommendation](https://mlsanity.com/paper/2604.12372/) (2026)
- [DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Trading](https://mlsanity.com/paper/2605.25527/) (2026)

## Abstract (arXiv)

Unlike traditional recommendation tasks, finite user time budgets introduce a critical resource constraint, requiring the recommender system to balance item relevance and evaluation cost. For example, in a mobile shopping interface, users interact with recommendations by scrolling, where each scroll triggers a list of items called slate. Users incur an evaluation cost - time spent assessing item features before deciding to click. Highly relevant items having higher evaluation costs may not fit within the user's time budget, affecting engagement. In this position paper, our objective is to evaluate reinforcement learning algorithms that learn patterns in user preferences and time budgets simultaneously, crafting recommendations with higher engagement potential under resource constraints. Our experiments explore the use of reinforcement learning to recommend items for users using Alibaba's Personalized Re-ranking dataset supporting slate optimization in e-commerce contexts. Our contributions include (i) a unified formulation of time-constrained slate recommendation modeled as Markov Decision Processes (MDPs) with budget-aware utilities; (ii) a simulation framework to study policy behavior on re-ranking data; and (iii) empirical evidence that on-policy and off-policy control can improve performance under tight time budgets than traditional contextual bandit-based methods.