# Interest-oriented Universal User Representation via Contrastive Learning

arXiv: https://arxiv.org/abs/2109.08865 · HTML: https://mlsanity.com/paper/2109.08865/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 18 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [Controllable Multi-Interest Framework for Recommendation](https://mlsanity.com/paper/2005.09347/) (2020)
- [PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest](https://mlsanity.com/paper/2007.03634/) (2020)
- [Modelling Context with User Embeddings for Sarcasm Detection in Social Media](https://mlsanity.com/paper/1607.00976/) (2016)
- [Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks](https://mlsanity.com/paper/1805.10727/) (2018)
- [PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervision](https://mlsanity.com/paper/2010.01494/) (2020)
- [Learning Invariant Representations of Social Media Users](https://mlsanity.com/paper/1910.04979/) (2019)
- [Exploiting Behavioral Consistence for Universal User Representation](https://mlsanity.com/paper/2012.06146/) (2020)
- [Calendar Graph Neural Networks for Modeling Time Structures in Spatiotemporal User Behaviors](https://mlsanity.com/paper/2006.06820/) (2020)

## Built on by (most cited first, 3 of 3)

- [Scaling Law for Recommendation Models: Towards General-purpose User Representations](https://mlsanity.com/paper/2111.11294/) (2021)
- [Task Relation-aware Continual User Representation Learning](https://mlsanity.com/paper/2306.01792/) (2023)
- [Empowering General-purpose User Representation with Full-life Cycle Behavior Modeling](https://mlsanity.com/paper/2110.11337/) (2021)

## Abstract (arXiv)

User representation is essential for providing high-quality commercial services in industry. Universal user representation has received many interests recently, with which we can be free from the cumbersome work of training a specific model for each downstream application. In this paper, we attempt to improve universal user representation from two points of views. First, a contrastive self-supervised learning paradigm is presented to guide the representation model training. It provides a unified framework that allows for long-term or short-term interest representation learning in a data-driven manner. Moreover, a novel multi-interest extraction module is presented. The module introduces an interest dictionary to capture principal interests of the given user, and then generate his/her interest-oriented representations via behavior aggregation. Experimental results demonstrate the effectiveness and applicability of the learned user representations.