mlsanityData to 7 Oct 2026

Online Personalizing White-box LLMs Generation with Neural Bandits

arXiv 2404.16115 ↗First posted 24 Apr 2024cs.CL cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

The advent of personalized content generation by LLMs presents a novel challenge: how to efficiently adapt text to meet individual preferences without the unsustainable demand of creating a unique model for each user. This study introduces an innovative online method that employs neural bandit algorithms to dynamically optimize soft instruction embeddings based on user feedback, enhancing the personalization of open-ended text generation by white-box LLMs. Through rigorous experimentation on various tasks, we demonstrate significant performance improvements over baseline strategies. NeuralTS, in particular, leads to substantial enhancements in personalized news headline generation, achieving up to a 62.9% improvement in terms of best ROUGE scores and up to 2.76% increase in LLM-agent evaluation against the baseline.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 24 Apr 2024.

Builds on

25 of 48 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
13 more
  1. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  2. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  3. 2017A Tutorial on Thompson Samplingcited by 420
  4. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  5. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  6. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  7. 2020Neural Thompson Samplingcited by 127
  8. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  9. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 77
  10. 2019Towards Controllable and Personalized Review Generationcited by 24
  11. 2021Idiosyncratic but not Arbitrary: Learning Idiolects in Online Registers Reveals Distinctive yet Consistent Individual Stylescited by 18
  12. 2020Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Networkcited by 14
  13. 2021Interaction-Grounded Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Prompt Optimization with Human Feedbackcited by 35
  2. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 25
  3. 2025Survey: Multi-Armed Bandits Meet Large Language Modelscited by 6
  4. 2026A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Banditscited by 1
  5. 2026Active Learning for Stochastic Contextual Linear Banditscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025T-POP: Test-Time Personalization with Online Preference Feedback
  2. 2026Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward
  3. 2025Personalized LLM Decoding via Contrasting Personal Preference
  4. 2023Parameter Efficient Tuning Allows Scalable Personalization of LLMs for Text Entry: A Case Study on Abbreviation Expansion
  5. 2024PAD: Personalized Alignment of LLMs at Decoding-Time
  6. 2025Iterative Critique-Refine Framework for Enhancing LLM Personalization
  7. 2024Orchestrating LLMs with Different Personalizations
  8. 2026Generate to Explore, Select to Exploit: Aligning LLM-based Headline Generation with Personalized Recommendation