# How Individual Traits and Language Styles Shape Preferences In Open-ended User-LLM Interaction: A Preliminary Study

arXiv: https://arxiv.org/abs/2504.17083 · HTML: https://mlsanity.com/paper/2504.17083/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [A Computational Approach to Politeness with Application to Social Factors](https://mlsanity.com/paper/1306.6078/) (2013)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions](https://mlsanity.com/paper/2308.02312/) (2023)
- [Investigating Affective Use and Emotional Well-being on ChatGPT](https://mlsanity.com/paper/2504.03888/) (2025)
- [Gibbs Sampling with People](https://mlsanity.com/paper/2008.02595/) (2020)
- [Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback](https://mlsanity.com/paper/2410.19133/) (2024)
- [The Future of Open Human Feedback](https://mlsanity.com/paper/2408.16961/) (2024)
- [Chatbot Companionship: A Mixed-Methods Study of Companion Chatbot Usage Patterns and Their Relationship to Loneliness in Active Users](https://mlsanity.com/paper/2410.21596/) (2024)

## Built on by (most cited first, 7 of 7)

- [Measuring and mitigating overreliance to build human-compatible AI](https://mlsanity.com/paper/2509.08010/) (2025)
- [Arch-Router: Aligning LLM Routing with Human Preferences](https://mlsanity.com/paper/2506.16655/) (2025)
- [GMTRouter: Personalized LLM Router over Multi-turn User Interactions](https://mlsanity.com/paper/2511.08590/) (2025)
- [A Concise Agent is Less Expert: Revealing Side Effects of Using Style Features on Conversational Agents](https://mlsanity.com/paper/2601.10809/) (2026)
- [From Performance to Purpose: A Sociotechnical Taxonomy for Evaluating Large Language Model Utility](https://mlsanity.com/paper/2602.20513/) (2026)
- [A Human-Centred Approach to Benchmarking LLMs for Parenting Advice](https://mlsanity.com/paper/2608.14622/) (2026)
- [Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts](https://mlsanity.com/paper/2609.38600/) (2026)

## Abstract (arXiv)

What makes an interaction with the LLM more preferable for the user? While it is intuitive to assume that information accuracy in the LLM's responses would be one of the influential variables, recent studies have found that inaccurate LLM's responses could still be preferable when they are perceived to be more authoritative, certain, well-articulated, or simply verbose. These variables interestingly fall under the broader category of language style, implying that the style in the LLM's responses might meaningfully influence users' preferences. This hypothesized dynamic could have double-edged consequences: enhancing the overall user experience while simultaneously increasing their susceptibility to risks such as LLM's misinformation or hallucinations. In this short paper, we present our preliminary studies in exploring this subject. Through a series of exploratory and experimental user studies, we found that LLM's language style does indeed influence user's preferences, but how and which language styles influence the preference varied across different user populations, and more interestingly, moderated by the user's very own individual traits. As a preliminary work, the findings in our studies should be interpreted with caution, particularly given the limitations in our samples, which still need wider demographic diversity and larger sample sizes. Our future directions will first aim to address these limitations, which would enable a more comprehensive joint effect analysis between the language style, individual traits, and preferences, and further investigate the potential causal relationship between and beyond these variables.