# Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

arXiv: https://arxiv.org/abs/2602.14492 · HTML: https://mlsanity.com/paper/2602.14492/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Builds on (most cited first, 25 of 25)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method](https://mlsanity.com/paper/2402.17193/) (2024)
- [Scaling Sentence Embeddings with Large Language Models](https://mlsanity.com/paper/2307.16645/) (2023)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks](https://mlsanity.com/paper/2511.07025/) (2025)
- [KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model](https://mlsanity.com/paper/2501.01028/) (2025)
- [Answer is All You Need: Instruction-following Text Embedding via Answering the Question](https://mlsanity.com/paper/2402.09642/) (2024)
- [Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learning](https://mlsanity.com/paper/2512.01282/) (2025)
- [Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta](https://mlsanity.com/paper/2311.09544/) (2023)
- [LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendation](https://mlsanity.com/paper/2506.21579/) (2025)
- [One4all User Representation for Recommender Systems in E-commerce](https://mlsanity.com/paper/2106.00573/) (2021)
- [Do We Need Domain-Specific Embedding Models? An Empirical Investigation](https://mlsanity.com/paper/2409.18511/) (2024)
- [Little Giants: Synthesizing High-Quality Embedding Data at Scale](https://mlsanity.com/paper/2410.18634/) (2024)
- [One for All, All for One: Learning and Transferring User Embeddings for Cross-Domain Recommendation](https://mlsanity.com/paper/2211.11964/) (2022)
- [Transferable and Forecastable User Targeting Foundation Model](https://mlsanity.com/paper/2412.12468/) (2024)
- [Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings](https://mlsanity.com/paper/2509.12892/) (2025)
- [Can One Embedding Fit All? A Multi-Interest Learning Paradigm Towards Improving User Interest Diversity Fairness](https://mlsanity.com/paper/2402.13495/) (2024)
- [How Do Decoder-Only LLMs Perceive Users? Rethinking Attention Masking for User Representation Learning](https://mlsanity.com/paper/2602.10622/) (2026)

## Built on by (most cited first, 1 of 1)

- [EchoRL: Reinforcement Learning via Rollout Echoing](https://mlsanity.com/paper/2605.31228/) (2026)

## Abstract (arXiv)

Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.