# Few-shot Personalization of LLMs with Mis-aligned Responses

arXiv: https://arxiv.org/abs/2406.18678 · HTML: https://mlsanity.com/paper/2406.18678/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 30 of 33)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Progressive Prompts: Continual Learning for Language Models](https://mlsanity.com/paper/2301.12314/) (2023)
- [Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets](https://mlsanity.com/paper/2106.10328/) (2021)
- [GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models](https://mlsanity.com/paper/2203.07281/) (2022)
- [Evaluating Open-Domain Question Answering in the Era of Large Language Models](https://mlsanity.com/paper/2305.06984/) (2023)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [Group Preference Optimization: Few-Shot Alignment of Large Language Models](https://mlsanity.com/paper/2310.11523/) (2023)
- [On the steerability of large language models toward data-driven personas](https://mlsanity.com/paper/2311.04978/) (2023)

## Built on by (most cited first, 30 of 32)

- [A Survey of Personalized Large Language Models: Progress and Future Directions](https://mlsanity.com/paper/2502.11528/) (2025)
- [Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://mlsanity.com/paper/2502.16761/) (2025)
- [Benchmarking Distributional Alignment of Large Language Models](https://mlsanity.com/paper/2411.05403/) (2024)
- [HumanLM: Simulating Users with State Alignment Beats Response Imitation](https://mlsanity.com/paper/2603.03303/) (2026)
- [A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models](https://mlsanity.com/paper/2504.07070/) (2025)
- [Instant Personalized Large Language Model Adaptation via Hypernetwork](https://mlsanity.com/paper/2510.16282/) (2025)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs](https://mlsanity.com/paper/2603.16557/) (2026)
- [Matryoshka Pilot: Learning to Drive Black-Box LLMs with LLMs](https://mlsanity.com/paper/2410.20749/) (2024)
- [Capturing Individual Human Preferences with Reward Features](https://mlsanity.com/paper/2503.17338/) (2025)
- [Personalized Language Models via Privacy-Preserving Evolutionary Model Merging](https://mlsanity.com/paper/2503.18008/) (2025)
- [T-POP: Test-Time Personalization with Online Preference Feedback](https://mlsanity.com/paper/2509.24696/) (2025)
- [Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning](https://mlsanity.com/paper/2502.08972/) (2025)
- [Training-free LLM Verification via Recycling Few-shot Examples](https://mlsanity.com/paper/2506.17251/) (2025)
- [RPM: Reasoning-Level Personalization for Black-Box Large Language Models](https://mlsanity.com/paper/2505.21082/) (2025)
- [Integrating Personality into Digital Humans: A Review of LLM-Driven Approaches for Virtual Reality](https://mlsanity.com/paper/2503.16457/) (2025)
- [AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys](https://mlsanity.com/paper/2511.07871/) (2025)
- [EMCEE: Improving Multilingual Capability of LLMs via Bridging Knowledge and Reasoning with Extracted Synthetic Multilingual Context](https://mlsanity.com/paper/2503.05846/) (2025)
- [Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?](https://mlsanity.com/paper/2505.13257/) (2025)
- [Iterative Critique-Refine Framework for Enhancing LLM Personalization](https://mlsanity.com/paper/2510.24469/) (2025)
- [Graph-Based Alternatives to LLMs for Human Simulation](https://mlsanity.com/paper/2511.02135/) (2025)
- [Designing Beyond Language: Sociotechnical Barriers in AI Health Technologies for Limited English Proficiency](https://mlsanity.com/paper/2511.07277/) (2025)
- [PRISP: Privacy-Safe Few-Shot Personalization via Lightweight Adaptation](https://mlsanity.com/paper/2601.06471/) (2026)
- [Batch Aggregation: An Approach to Enhance Text Classification with Correlated Augmented Data](https://mlsanity.com/paper/2504.05020/) (2025)
- [The Personalization Paradox: Semantic Loss vs. Reasoning Gains in Agentic AI Q&A](https://mlsanity.com/paper/2512.04343/) (2025)
- [Personalizing black-box models for nonparametric regression with minimax optimality](https://mlsanity.com/paper/2601.01432/) (2026)
- [Anamnesis: An Open-Source Platform for Large-Scale Backstory-Conditioned Survey Simulation](https://mlsanity.com/paper/2607.10628/) (2026)
- [PersonaEdit: Representative Sample Selection for Personalized Model Editing](https://mlsanity.com/paper/2608.27816/) (2026)
- [One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extraction](https://mlsanity.com/paper/2609.21626/) (2026)
- [TRACER: Trajectory-Aligned Learning for Multi-Turn User Simulation](https://mlsanity.com/paper/2609.28690/) (2026)

## Abstract (arXiv)

As the diversity of users increases, the capability of providing personalized responses by large language models (LLMs) has become increasingly important. Existing approaches have only limited successes in LLM personalization, due to the absence of personalized learning or the reliance on shared personal data. This paper proposes a new approach for a few-shot personalization of LLMs with their mis-aligned responses (Fermi). Our key idea is to learn a set of personalized prompts for each user by progressively improving the prompts using LLMs, based on user profile (e.g., demographic information) and a few examples of previous opinions. During an iterative process of prompt improvement, we incorporate the contexts of mis-aligned responses by LLMs, which are especially crucial for the effective personalization of LLMs. In addition, we develop an effective inference method to further leverage the context of the test query and the personalized prompts. Our experimental results demonstrate that Fermi significantly improves performance across various benchmarks, compared to best-performing baselines.