# Partially Observable Markov Decision Process for Recommender Systems

arXiv: https://arxiv.org/abs/1608.07793 · HTML: https://mlsanity.com/paper/1608.07793/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2016.

## Built on by (most cited first, 11 of 11)

- [Estimating and Penalizing Induced Preference Shifts in Recommender Systems](https://mlsanity.com/paper/2204.11966/) (2022)
- [Modeling Attrition in Recommender Systems with Departing Bandits](https://mlsanity.com/paper/2203.13423/) (2022)
- [Agentic Recommender System with Hierarchical Belief-State Memory](https://mlsanity.com/paper/2605.14401/) (2026)
- [Collaborative Policy Learning for Dynamic Scheduling Tasks in Cloud-Edge-Terminal IoT Networks Using Federated Reinforcement Learning](https://mlsanity.com/paper/2307.00541/) (2023)
- [RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems](https://mlsanity.com/paper/2508.00201/) (2025)
- [Emergent Instabilities in Algorithmic Feedback Loops](https://mlsanity.com/paper/2201.07203/) (2022)
- [Modeling Churn in Recommender Systems with Aggregated Preferences](https://mlsanity.com/paper/2502.18483/) (2025)
- [System-Agnostic Meta-Learning for MDP-based Dynamic Scheduling via Descriptive Policy](https://mlsanity.com/paper/2201.07051/) (2022)
- [Explicit User Manipulation in Reinforcement Learning Based Recommender Systems](https://mlsanity.com/paper/2203.10629/) (2022)
- [Churn-Aware Recommendation Planning under Aggregated Preference Feedback](https://mlsanity.com/paper/2507.04513/) (2025)
- [Factorized Scheduling Principle: Learning Interpretable and Transferable Policies via Structured Additive Functions](https://mlsanity.com/paper/2609.36578/) (2026)

## Abstract (arXiv)

We report the "Recurrent Deterioration" (RD) phenomenon observed in online recommender systems. The RD phenomenon is reflected by the trend of performance degradation when the recommendation model is always trained based on users' feedbacks of the previous recommendations. There are several reasons for the recommender systems to encounter the RD phenomenon, including the lack of negative training data and the evolution of users' interests, etc. Motivated to tackle the problems causing the RD phenomenon, we propose the POMDP-Rec framework, which is a neural-optimized Partially Observable Markov Decision Process algorithm for recommender systems. We show that the POMDP-Rec framework effectively uses the accumulated historical data from real-world recommender systems and automatically achieves comparable results with those models fine-tuned exhaustively by domain exports on public datasets.