mlsanityData to 14 Oct 2026

Subjective Behaviors and Preferences in LLM: Language of Browsing

arXiv 2508.15474 ↗First posted 21 Aug 2025cs.CL cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

A Large Language Model (LLM) offers versatility across domains and tasks, purportedly benefiting users with a wide variety of behaviors and preferences. We question this perception about an LLM when users have inherently subjective behaviors and preferences, as seen in their ubiquitous and idiosyncratic browsing of websites or apps. The sequential behavior logs of pages, thus generated, form something akin to each user's self-constructed "language", albeit without the structure and grammar imbued in natural languages. We ask: (i) Can a small LM represent the "language of browsing" better than a large LM? (ii) Can an LM with a single set of parameters (or, single LM) adequately capture myriad users' heterogeneous, subjective behaviors and preferences? (iii) Can a single LM with high average performance, yield low variance in performance to make alignment good at user level? We introduce clusterwise LM training, HeTLM (Heterogeneity aware Training of Language Model), appropriate for subjective behaviors. We find that (i) a small LM trained using a page-level tokenizer outperforms large pretrained or finetuned LMs; (ii) HeTLM with heterogeneous cluster specific set of parameters outperforms a single LM of the same family, controlling for the number of parameters; and (iii) a higher mean and a lower variance in generation ensues, implying improved alignment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 20 of its 30 references that are in the map. First posted on arXiv 21 Aug 2025.

Builds on

20 of 30 references.

References · 20

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  6. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  8. 2023A Survey on Large Language Models for Recommendationcited by 379
  9. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 299
  10. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  11. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 170
  12. 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
8 more
  1. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  2. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluationcited by 88
  3. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  4. 2018Clustering with Deep Learning: Taxonomy and New Methodscited by 46
  5. 2020Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognitioncited by 35
  6. 2020Temporal Phenotyping using Deep Predictive Clustering of Disease Progressioncited by 26
  7. 2023SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domainscited by 12
  8. 2019Predictive clusteringcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions
  2. 2024A Survey on Human Preference Learning for Large Language Models
  3. 2024Towards a Unified View of Preference Learning for Large Language Models: A Survey
  4. 2026Vela: User Behavior as Supervision Sparks LLM Intelligence
  5. 2026Personalized Benchmarking: Evaluating LLMs by Individual Preferences
  6. 2024Aligning LLMs with Individual Preferences via Interaction
  7. 2026Many Preferences, Few Policies: Towards Scalable Language Model Personalization
  8. 2025CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions