# Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models

arXiv: https://arxiv.org/abs/2505.17051 · HTML: https://mlsanity.com/paper/2505.17051/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 57 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 30 of 40)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging](https://mlsanity.com/paper/2310.11564/) (2023)
- [In-context Autoencoder for Context Compression in a Large Language Model](https://mlsanity.com/paper/2307.06945/) (2023)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [MaxMin-RLHF: Alignment with Diverse Human Preferences](https://mlsanity.com/paper/2402.08925/) (2024)
- [LMFusion: Adapting Pretrained Language Models for Multimodal Generation](https://mlsanity.com/paper/2412.15188/) (2024)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [Personalizing Dialogue Agents via Meta-Learning](https://mlsanity.com/paper/1905.10033/) (2019)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [PERSONA: A Reproducible Testbed for Pluralistic Alignment](https://mlsanity.com/paper/2407.17387/) (2024)
- [A Survey of Personalized Large Language Models: Progress and Future Directions](https://mlsanity.com/paper/2502.11528/) (2025)
- [HYDRA: Model Factorization Framework for Black-Box LLM Personalization](https://mlsanity.com/paper/2406.02888/) (2024)
- [Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations](https://mlsanity.com/paper/2306.08121/) (2023)
- [Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement](https://mlsanity.com/paper/2402.11060/) (2024)
- [Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation](https://mlsanity.com/paper/2204.08128/) (2022)
- [500xCompressor: Generalized Prompt Compression for Large Language Models](https://mlsanity.com/paper/2408.03094/) (2024)
- [XRec: Large Language Models for Explainable Recommendation](https://mlsanity.com/paper/2406.02377/) (2024)
- [LLMs + Persona-Plug = Personalized LLMs](https://mlsanity.com/paper/2409.11901/) (2024)
- [User Embedding Model for Personalized Language Prompting](https://mlsanity.com/paper/2401.04858/) (2024)
- [Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences](https://mlsanity.com/paper/2405.15065/) (2024)
- [ComPO: Community Preferences for Language Model Personalization](https://mlsanity.com/paper/2410.16027/) (2024)
- [Learning Retrieval Augmentation for Personalized Dialogue Generation](https://mlsanity.com/paper/2406.18847/) (2024)

## Built on by (most cited first, 10 of 10)

- [Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents](https://mlsanity.com/paper/2510.04491/) (2025)
- [A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges](https://mlsanity.com/paper/2507.21117/) (2025)
- [A Unified Language Model for Large Scale Search, Recommendation, and Reasoning](https://mlsanity.com/paper/2603.17533/) (2026)
- [ChildEval: When large language models meet children's personalities](https://mlsanity.com/paper/2605.27805/) (2026)
- [Latent Personal Memory: Represent personal memory as dynamic soft prompts](https://mlsanity.com/paper/2606.20911/) (2026)
- [PriorProbe: Recovering Individual-Level Priors for Personalizing Neural Networks in Facial Expression Recognition](https://mlsanity.com/paper/2602.03882/) (2026)
- [The Subjectivity of Respect in Police Traffic Stops: Modeling Community Perspectives in Body-Worn Camera Footage](https://mlsanity.com/paper/2602.10339/) (2026)
- [ClusterRAG: Cluster-Based Collaborative Filtering for Personalized Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.18769/) (2026)
- [Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport](https://mlsanity.com/paper/2608.11342/) (2026)
- [Attribute-Based Activation Steering of LLMs for Group-Specific Explanation Generation](https://mlsanity.com/paper/2608.29215/) (2026)

## Abstract (arXiv)

Large language models (LLMs) excel at generating contextually relevant content. However, tailoring these outputs to individual users for effective personalization is a significant challenge. While rich user-specific information often exists as pre-existing user representations, such as embeddings learned from preferences or behaviors, current methods to leverage these for LLM personalization typically require costly fine-tuning or token-heavy prompting. We propose Embedding-to-Prefix (E2P), a parameter-efficient method that injects pre-computed context embeddings into an LLM's hidden representation space through a learned projection to a single soft token prefix. This enables effective personalization while keeping the backbone model frozen and avoiding expensive adaptation techniques. We evaluate E2P across two public datasets and in a production setting: dialogue personalization on Persona-Chat, contextual headline generation on PENS, and large-scale personalization for music and podcast consumption. Results show that E2P preserves contextual signals and achieves strong performance with minimal computational overhead, offering a scalable, efficient solution for contextualizing generative AI systems.