mlsanityData to 14 Oct 2026

Reinforcement Learning with Conformal Action Sets: An Application to Sequential Recommendation

arXiv 2610.08743 ↗First posted 6 Oct 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models also fits: Deep RL algorithms and representations

Abstract (arXiv)

Sequential recommenders typically use a fixed slate size even though the number of useful alternatives changes within a session. We propose Reinforcement Learning with Calibrated Pruning (RLCP), which adapts the retained action set using critic scores and an online threshold. The threshold is updated from binary feedback indicating whether the set contains an action in a proxy target. We prove a deterministic bound on the observed proxy miss rate along adaptive trajectories. To quantify the effect of pruning on reward, we derive an exact decomposition of value loss into filtering and selection losses. Under explicit proxy and critic approximation conditions, this decomposition yields a finite session reward bound that also accounts for imperfect selection and set truncation, without requiring the learning parameters to converge. Experiments on KuaiRand-Pure and MovieLens 1M compare two RLCP implementations with four RL baselines. In each of the 19 configurations, at least one RLCP variant achieves the highest catalog diversity, reaching 1.11× to 5.21× that of the strongest baseline, with competitive session depth and no larger retained sets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 25 references that are in the map. First posted on arXiv 6 Oct 2026.

Builds on

5 of 25 references.

References · 5

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2022KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videoscited by 72
  3. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  4. 2025Conformal Constrained Policy Optimization for Cost-Effective LLM Agentscited by 7
  5. 2024Confidence-aware Fine-tuning of Sequential Recommendation Systems via Conformal Predictioncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Robust Reinforcement Learning Objectives for Sequential Recommender Systems
  2. 2026FlexRec: Adapting LLM-based Recommenders for Flexible Needs via Reinforcement Learning
  3. 2025Reward Balancing Revisited: Enhancing Offline Reinforcement Learning for Recommender Systems
  4. 2026ProRL: Effective Reinforcement Learning for Proactive Recommendation via Rectified Policy Gradient Estimation
  5. 2023Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation
  6. 2026Objective Shaping with Hard Negatives: Windowed Partial AUC Optimization for RL-based LLM Recommenders ↗
  7. 2024Retentive Decision Transformer with Adaptive Masking for Reinforcement Learning based Recommendation Systems
  8. 2024EasyRL4Rec: An Easy-to-use Library for Reinforcement Learning Based Recommender Systems