mlsanityData to 7 Oct 2026

Learning Personalized Alignment for Evaluating Open-ended Text Generation

arXiv 2310.03304 ↗First posted 5 Oct 2023cs.CLcited by 29

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Recent research has increasingly focused on evaluating large language models' (LLMs) alignment with diverse human values and preferences, particularly for open-ended tasks like story generation. Traditional evaluation metrics rely heavily on lexical similarity with human-written references, often showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. To address these challenges, we introduce PerSE, an interpretable evaluation framework designed to assess alignment with specific human preferences. It is tuned to infer specific preferences from an in-context personal profile and evaluate the alignment between the generated content and personal preferences. PerSE enhances interpretability by providing detailed comments and fine-grained scoring, facilitating more personalized content generation. Our 13B LLaMA-2-based PerSE shows a 15.8% increase in Kendall correlation and a 13.7% rise in accuracy with zero-shot reviewers compared to GPT-4. It also outperforms GPT-4 by 46.01% in Kendall correlation on new domains, indicating its transferability.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 64 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

32 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback9 Mar
  2. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  8. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  10. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  11. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  12. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
18 more
  1. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  2. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  3. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  4. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  5. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  6. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  7. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  8. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
  9. 2021OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metricscited by 61
  10. 2020UNION: An Unreferenced Metric for Evaluating Open-ended Story Generationcited by 60
  11. 2018A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generationcited by 42
  12. 2022StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoningcited by 21
  13. 2021Refocusing on Relevance: Personalization in NLGcited by 18
  14. 2022Rethinking Personalized Ranking at Pinterest: An End-to-End Approachcited by 16
  15. 2021Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generationcited by 13
  16. 2023Prompt Tuning Large Language Models on Personalized Aspect Extraction for Recommendationscited by 13
  17. 2020Narrative Text Generation with a Latent Discrete Plancited by 12
  18. 2018MPST: A Corpus of Movie Plot Synopses with Tagscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 19; 2026: 10.

29 papers.

Sort:

By area:

  1. 2025ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generationcited by 26
  2. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 25
  3. 2025Instant Personalized Large Language Model Adaptation via Hypernetworkcited by 15
  4. 2025A Cognitive Writing Perspective for Constrained Long-Form Text Generationcited by 14
  5. 2025Towards Personalized Deep Research: Benchmarks and Evaluationscited by 14
  6. 2025GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projectioncited by 13
  7. 2025Pluralistic Off-policy Evaluation and Alignmentcited by 11
  8. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 10
  9. 2025When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learningcited by 9
  10. 2025AI Realtor: Towards Grounded Persuasive Language Generation for Automated Copywritingcited by 6
  11. 2025PREF: Reference-Free Evaluation of Personalised Text Generation in LLMscited by 5
  12. 2025Benchmarking and Improving LLM Robustness for Personalized Generationcited by 4
17 more
  1. 2026P-GenRM: Personalized Generative Reward Model with Test-time User-based Scalingcited by 4
  2. 2025Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generationcited by 3
  3. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  4. 2025Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Stylecited by 1
  5. 2025Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?cited by 1
  6. 2025MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMscited by 1
  7. 2025CAP-LLM: Context-Augmented Personalized Large Language Models for News Headline Generationcited by 1
  8. 2025Psychologically Enhanced AI Agentscited by 1
  9. 2025Enhancing Rating Prediction with Off-the-Shelf LLMs Using In-Context User Reviewscited by 1
  10. 2025PREFINE: Personalized Story Generation via Simulated User Critics and User-Specific Rubric Generationcited by 1
  11. 2026ErrEval: Error-Aware Evaluation for Question Generation through Explicit Diagnosticscited by 1
  12. 2026StoryLens: Preference-Aligned Story Rewriting via Context-Aware Narrative Enrichmentcited by 1
  13. 2026Magic, Madness, Heaven, Sin: LLM Output Diversity is Everything, Everywhere, All at Oncecited by 0
  14. 2026NARRA-Gym for Evaluating Interactive Narrative Agentscited by 0
  15. 2026Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Modelscited by 0
  16. 2026PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspacescited by 0
  17. 2026Gradient-Aligned Pair Selection for Personalized Preference Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation
  2. 2024Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment
  3. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications
  4. 2025On Evaluating LLM Alignment by Evaluating LLMs as Judges
  5. 2024Uncovering Factor Level Preferences to Improve Human-Model Alignment
  6. 2025GRAVITY: A Framework for Personalized Text Generation via Profile-Grounded Synthetic Preferences
  7. 2024PAD: Personalized Alignment of LLMs at Decoding-Time
  8. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators