# Incorporating Group Prior into Variational Inference for Tail-User Behavior Modeling in CTR Prediction

arXiv: https://arxiv.org/abs/2410.15098 · HTML: https://mlsanity.com/paper/2410.15098/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 19 Oct 2024.

## Builds on (most cited first, 14 of 14)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09248/) (2019)
- [Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddings](https://mlsanity.com/paper/1904.11547/) (2019)
- [Deep Session Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.06482/) (2019)
- [Variational Attention for Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.08207/) (2017)
- [Sequential Scenario-Specific Meta Learner for Online Recommendation](https://mlsanity.com/paper/1906.00391/) (2019)
- [Two-Stage Constrained Actor-Critic for Short Video Recommendation](https://mlsanity.com/paper/2302.01680/) (2023)
- [Kalman Filtering Attention for User Behavior Modeling in CTR Prediction](https://mlsanity.com/paper/2010.00985/) (2020)
- [Alleviating Cold-start Problem in CTR Prediction with A Variational Embedding Learning Framework](https://mlsanity.com/paper/2201.10980/) (2022)

## Abstract (arXiv)

User behavior modeling -- which aims to extract user interests from behavioral data -- has shown great power in Click-through rate (CTR) prediction, a key component in recommendation systems. Recently, attention-based algorithms have become a promising direction, as attention mechanisms emphasize the relevant interactions from rich behaviors. However, the methods struggle to capture the preferences of tail users with sparse interaction histories. To address the problem, we propose a novel variational inference approach, namely Group Prior Sampler Variational Inference (GPSVI), which introduces group preferences as priors to refine latent user interests for tail users. In GPSVI, the extent of adjustments depends on the estimated uncertainty of individual preference modeling. In addition, We further enhance the expressive power of variational inference by a volume-preserving flow. An appealing property of the GPSVI method is its ability to revert to traditional attention for head users with rich behavioral data while consistently enhancing performance for long-tail users with sparse behaviors. Rigorous analysis and extensive experiments demonstrate that GPSVI consistently improves the performance of tail users. Moreover, online A/B testing on a large-scale real-world recommender system further confirms the effectiveness of our proposed approach.