mlsanityData to 7 Oct 2026

Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers

arXiv 2311.09180 ↗First posted 15 Nov 2023cs.CL cs.HC cs.IRcited by 91

Not on the map. Why?

Abstract (arXiv)

Powerful large language models have facilitated the development of writing assistants that promise to significantly improve the quality and efficiency of composition and communication. However, a barrier to effective assistance is the lack of personalization in LLM outputs to the author's communication style, specialized knowledge, and values. In this paper, we address this challenge by proposing Pearl, a LLM writing assistant personalized with a retriever that is trained to be generation-calibrated for personalization. Generation calibration ensures that our retriever selects historic user authored documents to augment an LLM prompt such that they are likely to help an LLM generation better adhere to a users' preferences. We propose two key novelties for training such a retriever: (1) A training data selection method that identifies user requests likely to benefit from personalization and documents that provide that benefit; and (2) A scale-calibrating KL-divergence objective that ensures that our retriever scores remain proportional to the downstream generation quality from using the document for personalized generation. In a series of holistic evaluations, we demonstrate the effectiveness of Pearl in generating long-form texts on multiple social media datasets. Finally, we demonstrate how a generation-calibrated retriever can double as a performance predictor -- detecting low quality retrieval, and improving potentially under-performing outputs via revision with LLMs.

Key facts. Cited by 91 papers in the map (data through 7 Oct 2026). Builds on 48 of its 83 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

48 of 83 references.

References · 48

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  6. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  9. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  10. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  11. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  12. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
36 more
  1. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  2. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  3. 2020Evaluation of Text Generation: A Surveycited by 275
  4. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  5. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  6. 2022Active Example Selection for In-Context Learningcited by 195
  7. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  8. 2022Large Language Models with Controllable Working Memorycited by 169
  9. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  10. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  11. 2018Training Millions of Personalized Dialogue Agentscited by 126
  12. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 97
  13. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  14. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  15. 2022Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writingcited by 71
  16. 2023Coverage-based Example Selection for In-Context Learningcited by 58
  17. 2016Smart Reply: Automated Response Suggestion for Emailcited by 54
  18. 2019Generating Personalized Recipes from Historical User Preferencescited by 52
  19. 2023ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfercited by 49
  20. 2023EcoAssistant: Using LLM Assistant More Affordably and Accuratelycited by 47
  21. 2020Automated Lay Language Summarization of Biomedical Scientific Reviewscited by 46
  22. 2022Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Modelscited by 24
  23. 2022Unifying Data Perspectivism and Personalization: An Application to Social Normscited by 22
  24. 2023Beyond Summarization: Designing AI Support for Real-World Expository Writing Taskscited by 20
  25. 2023Beyond Prompts: Exploring the Design Space of Mixed-Initiative Co-Creativity Systemscited by 20
  26. 2021Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Modelscited by 18
  27. 2023SCREWS: A Modular Framework for Reasoning with Revisionscited by 16
  28. 2021IGA : An Intent-Guided Authoring Assistantcited by 12
  29. 2022Calibration of Machine Reading Systems at Scalecited by 11
  30. 2020Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1cited by 10
  31. 2022ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Conceptscited by 10
  32. 2022Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorizationcited by 10
  33. 2023Writing Tools: Looking Back to Look Aheadcited by 8
  34. 2023Automatic and Human-AI Interactive Text Generationcited by 4
  35. 2023Multivariate Representation Learning for Information Retrievalcited by 3
  36. 2023Starting Conversations with Search Engines -- Interfaces that Elicit Natural Language Queriescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 19; 2025: 35; 2026: 36.

91 papers.

Sort:

By area:

  1. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  2. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  3. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  4. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  5. 2024LongLaMP: A Benchmark for Personalized Long-form Text Generationcited by 91
  6. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  7. 2024Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Effortscited by 71
  8. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 66
  9. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
  10. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  11. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 44
  12. 2024Can Language Models Reason about Individualistic Human Values and Preferences?cited by 34
38 more
  1. 2024AI PERSONA: Towards Life-long Personalization of LLMscited by 30
  2. 2025LaMP-QA: A Benchmark for Personalized Long-form Question Answeringcited by 30
  3. 2025Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generationcited by 29
  4. 2025Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generationcited by 26
  5. 2025ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generationcited by 26
  6. 2025A Survey of Personalization: From RAG to Agentcited by 26
  7. 2024SocialMind: LLM-based Proactive AR Social Assistive System with Human-like Perception for In-situ Live Interactionscited by 25
  8. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 25
  9. 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wildcited by 24
  10. 2025Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generationcited by 23
  11. 2024On the Way to LLM Personalization: Learning to Remember User Conversationscited by 21
  12. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Modelscited by 20
  13. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 20
  14. 2024GhostWriter: Augmenting Collaborative Human-AI Writing Experiences Through Personalization and Agencycited by 18
  15. 2025Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysiscited by 16
  16. 2024Navigating the Path of Writing: Outline-guided Text Generation with Large Language Modelscited by 15
  17. 2025Instant Personalized Large Language Model Adaptation via Hypernetworkcited by 15
  18. 2024Social Skill Training with Large Language Modelscited by 13
  19. 2025PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalizationcited by 13
  20. 2025The Era of Real-World Human Interaction: RL from User Conversationscited by 13
  21. 2024Think Together and Work Better: Combining Humans' and LLMs' Think-Aloud Outcomes for Effective Text Evaluationcited by 12
  22. 2025Personalized LLM Decoding via Contrasting Personal Preferencecited by 12
  23. 2025CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steeringcited by 12
  24. 2026VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactionscited by 11
  25. 2025EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devicescited by 10
  26. 2025Catch Me If You Can? Not Yet: LLMs Still Struggle to Imitate the Implicit Writing Styles of Everyday Authorscited by 10
  27. 2023RoleCraft-GLM: Advancing Personalized Role-Playing in Large Language Modelscited by 9
  28. 2024LLMExplainer: Large Language Model based Bayesian Inference for Graph Explanation Generationcited by 8
  29. 2025Personalized Language Models via Privacy-Preserving Evolutionary Model Mergingcited by 8
  30. 2025PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimizationcited by 8
  31. 2026BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMscited by 7
  32. 2024Language Models as Critical Thinking Tools: A Case Study of Philosopherscited by 6
  33. 2025T-POP: Test-Time Personalization with Online Preference Feedbackcited by 6
  34. 2025Learning from Natural Language Feedback for Personalized Question Answeringcited by 5
  35. 2025PaperDebugger: A Plugin-Based Multi-Agent System for In-Editor Academic Writing, Review, and Editingcited by 5
  36. 2025Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learningcited by 4
  37. 2025Personas within Parameters: Fine-Tuning Small Language Models with Low-Rank Adapters to Mimic User Behaviorscited by 4
  38. 2026Your Language Model Secretly Contains Personality Subnetworkscited by 4

50 most cited of 91 All 91 (JSON)