mlsanityData to 7 Oct 2026

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data

arXiv 2508.04698 ↗First posted 6 Aug 2025cs.CLcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

LLM-powered conversational assistants are often deployed in a one-size-fits-all manner, which fails to accommodate individual user preferences. Recently, LLM personalization -- tailoring models to align with specific user preferences -- has gained increasing attention as a way to bridge this gap. In this work, we specifically focus on a practical yet challenging setting where only a small set of preference annotations can be collected per user -- a problem we define as Personalized Preference Alignment with Limited Data (PPALLI). To support research in this area, we introduce two datasets -- DnD and ELIP -- and benchmark a variety of alignment techniques on them. We further propose FaST, a highly parameter-efficient approach that leverages high-level features automatically discovered from the data, achieving the best overall performance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 33 of its 48 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

33 of 48 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications21 Mar
  2. 2025Drift: Decoding-time Personalized Alignments with Implicit User Preferences20 Feb
  3. 2024PersonalLLM: Tailoring LLMs to Individual Preferences30 Sep
  4. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignment24 Jul
  5. 2024Orchestrating LLMs with Different Personalizations4 Jul
  6. 2024Personalized Language Modeling from Personalized Human Feedback6 Feb
  7. 2023Compositional preference models for aligning LMs17 Oct
  8. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging17 Oct

Other references · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  8. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  9. 2019ELI5: Long Form Question Answeringcited by 567
  10. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  11. 2024Direct Language Model Alignment from Online AI Feedbackcited by 257
  12. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
13 more
  1. 2024Many-Shot In-Context Learningcited by 203
  2. 2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 145
  3. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 142
  4. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforcecited by 141
  5. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 66
  6. 2022Dungeons and Dragons as a Dialog Challenge for Artificial Intelligencecited by 35
  7. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
  8. 2024Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignmentcited by 27
  9. 2023Data-Efficient Alignment of Large Language Models with Human Feedback Through Natural Languagecited by 22
  10. 2024DELIFT: Data Efficient Language model Instruction Fine Tuningcited by 14
  11. 2024ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Modelscited by 9
  12. 2025SPRI: Aligning Large Language Models with Context-Situated Principlescited by 5
  13. 2024ALMA: Alignment with Minimal Annotationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026LLM Consumer Behavior Theory: Foundations of a Novel Research Fieldcited by 0
  2. 2026Cautious Context Steering for Language Model Personalizationcited by 0
  3. 2026hoBIT: A Profile-Aware Retrieval-Augmented Chatbot for University Academic Advisingcited by 0
  4. 2026Ready to Speak: Aligning LLMs for TTS-Friendly Text Generationcited by 0
  5. 2026From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment
  2. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models
  3. 2024Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment
  4. 2025Group Preference Alignment: Customized LLM Response Generation from In-Situ Conversations
  5. 2024Aligning LLMs with Individual Preferences via Interaction
  6. 2025CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions
  7. 2024Aligning to Thousands of Preferences via System Message Generalization
  8. 2025HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation