# Exploration and Regularization of the Latent Action Space in Recommendation

arXiv: https://arxiv.org/abs/2302.03431 · HTML: https://mlsanity.com/paper/2302.03431/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2023.

## Built on by (most cited first, 17 of 17)

- [Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning](https://mlsanity.com/paper/2503.24289/) (2025)
- [A Survey of Personalization: From RAG to Agent](https://mlsanity.com/paper/2504.10147/) (2025)
- [Generative Auto-Bidding with Value-Guided Explorations](https://mlsanity.com/paper/2504.14587/) (2025)
- [Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-agent LLM](https://mlsanity.com/paper/2312.15450/) (2023)
- [SIGMA: Selective Gated Mamba for Sequential Recommendation](https://mlsanity.com/paper/2408.11451/) (2024)
- [EasyRL4Rec: An Easy-to-use Library for Reinforcement Learning Based Recommender Systems](https://mlsanity.com/paper/2402.15164/) (2024)
- [AURO: Reinforcement Learning for Adaptive User Retention Optimization in Recommender Systems](https://mlsanity.com/paper/2310.03984/) (2023)
- [Efficient and Robust Regularized Federated Recommendation](https://mlsanity.com/paper/2411.01540/) (2024)
- [Exploring Recommender System Evaluation: A Multi-Modal User Agent Framework for A/B Testing](https://mlsanity.com/paper/2601.04554/) (2026)
- [UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systems](https://mlsanity.com/paper/2401.09034/) (2024)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Learning a Single Token to Replace Long System Prompts in LLMs](https://mlsanity.com/paper/2511.23271/) (2025)
- [Owen-Shapley Policy Optimization: A Principled RL Algorithm for Generative Search LLMs](https://mlsanity.com/paper/2601.08403/) (2026)
- [LLM-Enhanced Reinforcement Learning for Long-Term User Satisfaction in Interactive Recommendation](https://mlsanity.com/paper/2601.19585/) (2026)
- [Supervised Learning-enhanced Multi-Group Actor Critic for Live Stream Allocation in Feed](https://mlsanity.com/paper/2412.10381/) (2024)
- [KLAN: Kuaishou Landing-page Adaptive Navigator](https://mlsanity.com/paper/2507.23459/) (2025)
- [Empowering Denoising Sequential Recommendation with Large Language Model Embeddings](https://mlsanity.com/paper/2510.04239/) (2025)

## Abstract (arXiv)

In recommender systems, reinforcement learning solutions have effectively boosted recommendation performance because of their ability to capture long-term user-system interaction. However, the action space of the recommendation policy is a list of items, which could be extremely large with a dynamic candidate item pool. To overcome this challenge, we propose a hyper-actor and critic learning framework where the policy decomposes the item list generation process into a hyper-action inference step and an effect-action selection step. The first step maps the given state space into a vectorized hyper-action space, and the second step selects the item list based on the hyper-action. In order to regulate the discrepancy between the two action spaces, we design an alignment module along with a kernel mapping function for items to ensure inference accuracy and include a supervision module to stabilize the learning process. We build simulated environments on public datasets and empirically show that our framework is superior in recommendation compared to standard RL baselines.