# Transferable and Forecastable User Targeting Foundation Model

arXiv: https://arxiv.org/abs/2412.12468 · HTML: https://mlsanity.com/paper/2412.12468/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Prompt-aligned Gradient for Prompt Tuning](https://mlsanity.com/paper/2205.14865/) (2022)
- [Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?](https://mlsanity.com/paper/2308.10168/) (2023)
- [Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey](https://mlsanity.com/paper/2302.10035/) (2023)
- [Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting](https://mlsanity.com/paper/2305.03324/) (2023)
- [Multi-Modal Self-Supervised Learning for Recommendation](https://mlsanity.com/paper/2302.10632/) (2023)
- [ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings](https://mlsanity.com/paper/2305.14321/) (2023)
- [Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks](https://mlsanity.com/paper/1805.10727/) (2018)
- [Exploiting Behavioral Consistence for Universal User Representation](https://mlsanity.com/paper/2012.06146/) (2020)
- [General-Purpose User Embeddings based on Mobile App Usage](https://mlsanity.com/paper/2005.13303/) (2020)
- [Pre-trained Token-replaced Detection Model as Few-shot Learner](https://mlsanity.com/paper/2203.03235/) (2022)
- [Know Your Needs Better: Towards Structured Understanding of Marketer Demands with Analogical Reasoning Augmented LLMs](https://mlsanity.com/paper/2401.04319/) (2024)

## Built on by (most cited first, 4 of 4)

- [FOUNDv2: Learning Unified User Quantized Tokenizers for User Representation](https://mlsanity.com/paper/2508.00956/) (2025)
- [Instruction-aware User Embedding via Synergistic Language and Representation Modeling](https://mlsanity.com/paper/2510.11016/) (2025)
- [Query as Anchor: Scenario-Adaptive User Representation via Large Language Model](https://mlsanity.com/paper/2602.14492/) (2026)
- [Towards a Densing Law for User Representation Learning at Billion-Scale Capacity](https://mlsanity.com/paper/2608.23392/) (2026)

## Abstract (arXiv)

User targeting, the process of selecting targeted users from a pool of candidates for non-expert marketers, has garnered substantial attention with the advancements in digital marketing. However, existing user targeting methods encounter two significant challenges: (i) Poor cross-domain and cross-scenario transferability and generalization, and (ii) Insufficient forecastability in real-world applications. These limitations hinder their applicability across diverse industrial scenarios. In this work, we propose FOUND, an industrial-grade, transferable, and forecastable user targeting foundation model. To enhance cross-domain transferability, our framework integrates heterogeneous multi-scenario user data, aligning them with one-sentence targeting demand inputs through contrastive pre-training. For improved forecastability, the text description of each user is derived based on anticipated future behaviors, while user representations are constructed from historical information. Experimental results demonstrate that our approach significantly outperforms existing baselines in cross-domain, real-world user targeting scenarios, showcasing the superior capabilities of FOUND. Moreover, our method has been successfully deployed on the Alipay platform and is widely utilized across various scenarios.