mlsanityData to 14 Oct 2026

User Embedding Model for Personalized Language Prompting

arXiv 2401.04858 ↗First posted 10 Jan 2024cs.CL cs.AI cs.IRcited by 28

Not on the map. Why?

Abstract (arXiv)

Modeling long histories plays a pivotal role in enhancing recommendation systems, allowing to capture user's evolving preferences, resulting in more precise and personalized recommendations. In this study we tackle the challenges of modeling long user histories for preference understanding in natural language. Specifically, we introduce a new User Embedding Module (UEM) that efficiently processes user history in free-form text by compressing and representing them as embeddings, to use them as soft prompts to a LM. Our experiments demonstrate the superior capability of this approach in handling significantly longer histories compared to conventional text based prompting methods, yielding substantial improvements in predictive performance. The main contribution of this research is to demonstrate the ability to bias language models with user signals represented as embeddings.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

19 of 31 references.

References · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  11. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  12. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
7 more
  1. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  2. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  3. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  4. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  5. 2023Learning to Rewrite Prompts for Personalized Text Generationcited by 40
  6. 2021UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysiscited by 28
  7. 2022Human Language Modelingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 7; 2026: 13.

28 papers.

Sort:

By area:

  1. 2024User-LLM: Efficient LLM Contextualization with User Embeddingscited by 76
  2. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 71
  3. 2024LLMs + Persona-Plug = Personalized LLMscited by 32
  4. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 21
  5. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  6. 2024Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrievalcited by 13
  7. 2024RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMscited by 13
  8. 2024Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Reviewcited by 12
  9. 2024Understanding the Role of User Profile in the Personalization of Large Language Modelscited by 11
  10. 2025Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Modelscited by 10
  11. 2025Modeling Student Learning with 3.8 Million Program Tracescited by 10
  12. 2024Beyond Retrieval: Generating Narratives in Conversational Recommender Systemscited by 8
16 more
  1. 2024From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Modelscited by 6
  2. 2025The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMscited by 6
  3. 2026CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMscited by 4
  4. 2026Exploiting contextual information to improve stance detection in informal political discourse with LLMscited by 2
  5. 2026Uncertainty-Aware Variational Reward Factorization via Probabilistic Preference Bases for LLM Personalizationcited by 1
  6. 2026Beyond Isolated Behaviors: Hierarchical User Modeling for LLM Personalizationcited by 1
  7. 2026Retrieval-Augmented Personalization with Foundation Models for Wearable Stress Detectioncited by 1
  8. 2025Interpretable Mnemonic Generation for Kanji Learning via Expectation-Maximizationcited by 0
  9. 2026The Subjectivity of Respect in Police Traffic Stops: Modeling Community Perspectives in Body-Worn Camera Footagecited by 0
  10. 2026IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inferencecited by 0
  11. 2026Efficient Personalization of Generative User Interfacescited by 0
  12. 2026Bridging Textual Profiles and Latent User Embeddings for Personalizationcited by 0
  13. 2026Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferencescited by 0
  14. 2026PersonaEdit: Representative Sample Selection for Personalized Model Editingcited by 0
  15. 2026Attribute-Based Activation Steering of LLMs for Group-Specific Explanation Generationcited by 0
  16. 2026Inferring Hidden User Models from the Behavior of Personalized LLM Agentscited by 0