# Adap-τ: Adaptively Modulating Embedding Magnitude for Recommendation

arXiv: https://arxiv.org/abs/2302.04775 · HTML: https://mlsanity.com/paper/2302.04775/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Neural Factorization Machines for Sparse Predictive Analytics](https://mlsanity.com/paper/1708.05027/) (2017)
- [Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2112.08679/) (2021)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [DeepInf: Social Influence Prediction with Deep Learning](https://mlsanity.com/paper/1807.05560/) (2018)
- [Learning Disentangled Representations for Recommendation](https://mlsanity.com/paper/1910.14238/) (2019)
- [Fast Matrix Factorization for Online Recommendation with Implicit Feedback](https://mlsanity.com/paper/1708.05024/) (2017)
- [Embarrassingly Shallow Autoencoders for Sparse Data](https://mlsanity.com/paper/1905.03375/) (2019)
- [Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning](https://mlsanity.com/paper/2202.06200/) (2022)
- [SimpleX: A Simple and Strong Baseline for Collaborative Filtering](https://mlsanity.com/paper/2109.12613/) (2021)
- [On the Effectiveness of Sampled Softmax Loss for Item Recommendation](https://mlsanity.com/paper/2201.02327/) (2022)
- [Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems](https://mlsanity.com/paper/2005.12964/) (2020)
- [Graph Convolutional Network for Recommendation with Low-pass Collaborative Filters](https://mlsanity.com/paper/2006.15516/) (2020)
- [Multi-Sample based Contrastive Loss for Top-k Recommendation](https://mlsanity.com/paper/2109.00217/) (2021)

## Built on by (most cited first, 19 of 19)

- [A Survey on Popularity Bias in Recommender Systems](https://mlsanity.com/paper/2308.01118/) (2023)
- [Large Language Models are Learnable Planners for Long-Term Recommendation](https://mlsanity.com/paper/2403.00843/) (2024)
- [How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective](https://mlsanity.com/paper/2404.12008/) (2024)
- [Language Representations Can be What Recommenders Need: Findings and Potentials](https://mlsanity.com/paper/2407.05441/) (2024)
- [Popularity-Aware Alignment and Contrast for Mitigating Popularity Bias](https://mlsanity.com/paper/2405.20718/) (2024)
- [Preference Diffusion for Recommendation](https://mlsanity.com/paper/2410.13117/) (2024)
- [PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation](https://mlsanity.com/paper/2411.00163/) (2024)
- [Intersectional Two-sided Fairness in Recommendation](https://mlsanity.com/paper/2402.02816/) (2024)
- [Graph Spectral Filtering with Chebyshev Interpolation for Recommendation](https://mlsanity.com/paper/2505.00552/) (2025)
- [Temporal Linear Item-Item Model for Sequential Recommendation](https://mlsanity.com/paper/2412.07382/) (2024)
- [Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling](https://mlsanity.com/paper/2507.00518/) (2025)
- [CDR: Conservative Doubly Robust Learning for Debiased Recommendation](https://mlsanity.com/paper/2308.08461/) (2023)
- [Exploring Heterogeneity and Uncertainty for Graph-based Cognitive Diagnosis Models in Intelligent Education](https://mlsanity.com/paper/2403.05559/) (2024)
- [On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective](https://mlsanity.com/paper/2505.11318/) (2025)
- [Macro Graph of Experts for Billion-Scale Multi-Task Recommendation](https://mlsanity.com/paper/2506.10520/) (2025)
- [Advancing Loss Functions in Recommender Systems: A Comparative Study with a R'enyi Divergence-Based Solution](https://mlsanity.com/paper/2506.15120/) (2025)
- [ALNSynergy: a graph convolutional network with multi-representation alignment for drug synergy prediction](https://mlsanity.com/paper/2311.16207/) (2023)
- [Talos: Optimizing Top-K Accuracy in Recommender Systems](https://mlsanity.com/paper/2601.19276/) (2026)
- [Correct and Weight: A Simple Yet Effective Loss for Implicit Feedback Recommendation](https://mlsanity.com/paper/2601.04291/) (2026)

## Abstract (arXiv)

Recent years have witnessed the great successes of embedding-based methods in recommender systems. Despite their decent performance, we argue one potential limitation of these methods -- the embedding magnitude has not been explicitly modulated, which may aggravate popularity bias and training instability, hindering the model from making a good recommendation. It motivates us to leverage the embedding normalization in recommendation. By normalizing user/item embeddings to a specific value, we empirically observe impressive performance gains (9% on average) on four real-world datasets. Although encouraging, we also reveal a serious limitation when applying normalization in recommendation -- the performance is highly sensitive to the choice of the temperature τ which controls the scale of the normalized embeddings. To fully foster the merits of the normalization while circumvent its limitation, this work studied on how to adaptively set the proper τ. Towards this end, we first make a comprehensive analyses of τ to fully understand its role on recommendation. We then accordingly develop an adaptive fine-grained strategy Adap-τ for the temperature with satisfying four desirable properties including adaptivity, personalized, efficiency and model-agnostic. Extensive experiments have been conducted to validate the effectiveness of the proposal. The code is available at urlhttps://github.com/junkangwu/Adaptau.