# Personalized Reward Learning with Interaction-Grounded Learning (IGL)

arXiv: https://arxiv.org/abs/2211.15823 · HTML: https://mlsanity.com/paper/2211.15823/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 78 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 21 of 21)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Learning from positive and unlabeled data: a survey](https://mlsanity.com/paper/1811.04820/) (2018)
- [Online Learning: A Comprehensive Survey](https://mlsanity.com/paper/1802.02871/) (2018)
- [Self-Imitation Learning](https://mlsanity.com/paper/1806.05635/) (2018)
- [Robust Reinforcement Learning using Offline Data](https://mlsanity.com/paper/2208.05129/) (2022)
- [A Contextual Bandit Bake-off](https://mlsanity.com/paper/1802.04064/) (2018)
- [Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue](https://mlsanity.com/paper/2009.09945/) (2020)
- [Partial Label Learning with Self-Guided Retraining](https://mlsanity.com/paper/1902.03045/) (2019)
- [A Survey on Contextual Multi-armed Bandits](https://mlsanity.com/paper/1508.03326/) (2015)
- [Understanding Teacher Gaze Patterns for Robot Learning](https://mlsanity.com/paper/1907.07202/) (2019)
- [Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness](https://mlsanity.com/paper/2110.08353/) (2021)
- [Interaction-Grounded Learning](https://mlsanity.com/paper/2106.04887/) (2021)
- [Measuring the Diversity of Facebook Reactions to Research](https://mlsanity.com/paper/2001.01029/) (2020)
- [Interaction-Grounded Learning with Action-inclusive Feedback](https://mlsanity.com/paper/2206.08364/) (2022)
- [Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference](https://mlsanity.com/paper/2105.00822/) (2021)
- [Investigating Potential Factors Associated with Gender Discrimination in Collaborative Recommender Systems](https://mlsanity.com/paper/2002.07786/) (2020)
- [X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback](https://mlsanity.com/paper/2203.02072/) (2022)
- [Understanding Acoustic Patterns of Human Teachers Demonstrating Manipulation Tasks to Robots](https://mlsanity.com/paper/2211.00352/) (2022)

## Built on by (most cited first, 9 of 9)

- [Fine-Tuning Language Models with Advantage-Induced Policy Alignment](https://mlsanity.com/paper/2306.02231/) (2023)
- [The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2311.00168/) (2023)
- [On the Way to LLM Personalization: Learning to Remember User Conversations](https://mlsanity.com/paper/2411.13405/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Choosing the Right Weights: Balancing Value, Strategy, and Noise in Recommender Systems](https://mlsanity.com/paper/2305.17428/) (2023)
- [P-GenRM: Personalized Generative Reward Model with Test-time User-based Scaling](https://mlsanity.com/paper/2602.12116/) (2026)
- [Provably Efficient Interactive-Grounded Learning with Personalized Reward](https://mlsanity.com/paper/2405.20677/) (2024)
- [An Information Theoretic Approach to Interaction-Grounded Learning](https://mlsanity.com/paper/2401.05015/) (2024)
- [Interaction-Grounded Learning for Contextual Markov Decision Processes with Personalized Feedback](https://mlsanity.com/paper/2602.08307/) (2026)

## Abstract (arXiv)

In an era of countless content offerings, recommender systems alleviate information overload by providing users with personalized content suggestions. Due to the scarcity of explicit user feedback, modern recommender systems typically optimize for the same fixed combination of implicit feedback signals across all users. However, this approach disregards a growing body of work highlighting that (i) implicit signals can be used by users in diverse ways, signaling anything from satisfaction to active dislike, and (ii) different users communicate preferences in different ways. We propose applying the recent Interaction Grounded Learning (IGL) paradigm to address the challenge of learning representations of diverse user communication modalities. Rather than requiring a fixed, human-designed reward function, IGL is able to learn personalized reward functions for different users and then optimize directly for the latent user satisfaction. We demonstrate the success of IGL with experiments using simulations as well as with real-world production traces.