mlsanityData to 7 Oct 2026

Interaction-Grounded Learning for Contextual Markov Decision Processes with Personalized Feedback

arXiv 2602.08307 ↗First posted 9 Feb 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we study Interaction-Grounded Learning (IGL) [Xie et al., 2021], a paradigm designed for realistic scenarios where the learner receives indirect feedback generated by an unknown mechanism, rather than explicit numerical rewards. While prior work on IGL provides efficient algorithms with provable guarantees, those results are confined to single-step settings, restricting their applicability to modern sequential decision-making systems such as multi-turn Large Language Model (LLM) deployments. To bridge this gap, we propose a computationally efficient algorithm that achieves a sublinear regret guarantee for contextual episodic Markov Decision Processes (MDPs) with personalized feedback. Technically, we extend the reward-estimator construction of Zhang et al. [2024a] from the single-step to the multi-step setting, addressing the unique challenges of decoding latent rewards under MDPs. Building on this estimator, we design an Inverse-Gap-Weighting (IGW) algorithm for policy optimization. Finally, we demonstrate the effectiveness of our method in learning personalized objectives from multi-turn interactions through experiments on both a synthetic episodic MDP and a real-world user booking dataset.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 35 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

29 of 35 references.

References · 29

  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 200
  3. 2020Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oraclescited by 195
  4. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 194
  5. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  6. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
  7. 2016Learning End-to-End Goal-Oriented Dialogcited by 104
  8. 2018Practical Contextual Bandits with Regression Oraclescited by 92
  9. 2018Building a Conversational Agent Overnight with Dialogue Self-Playcited by 70
  10. 2021Efficient First-Order Contextual Bandits: Prediction, Allocation, and Triangular Discriminationcited by 53
  11. 2020Information Directed Sampling for Linear Partial Monitoringcited by 39
  12. 2024Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Modelscited by 19
17 more
  1. 2019Accelerated Robot Learning via Human Brain Signalscited by 13
  2. 2021Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectivenesscited by 12
  3. 2022Optimism in Face of a Context: Regret Guarantees for Stochastic Contextual MDPcited by 11
  4. 2018Contextual bandits with surrogate losses: Margin bounds and efficient algorithmscited by 10
  5. 2022Learning Efficiently Function Approximation for Contextual MDPcited by 10
  6. 2023Practical Contextual Bandits with Feedback Graphscited by 10
  7. 2025Formalizing Learning from Language Feedback with Provable Guaranteescited by 10
  8. 2021Interaction-Grounded Learningcited by 9
  9. 2022Eluder-based Regret for Stochastic Contextual MDPscited by 9
  10. 2022Personalized Reward Learning with Interaction-Grounded Learning (IGL)cited by 9
  11. 2023Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximationcited by 9
  12. 2024Review-based Recommender Systems: A Survey of Approaches, Challenges and Future Perspectivescited by 8
  13. 2022Interaction-Grounded Learning with Action-inclusive Feedbackcited by 7
  14. 2024Efficient Contextual Bandits with Uninformed Feedback Graphscited by 6
  15. 2024Using LLMs to Investigate Correlations of Conversational Follow-up Queries with User Satisfactioncited by 6
  16. 2024Provably Efficient Interactive-Grounded Learning with Personalized Rewardcited by 3
  17. 2024An Information Theoretic Approach to Interaction-Grounded Learningcited by 2

Built on by

Not cited yet.