mlsanityData to 7 Oct 2026

HumT DumT: Measuring and controlling human-like language in LLMs

arXiv 2502.13259 ↗First posted 18 Feb 2025cs.CL cs.AI cs.CYcited by 11

Not on the map. Why?

Abstract (arXiv)

Should LLMs generate language that makes them seem human? Human-like language might improve user experience, but might also lead to deception, overreliance, and stereotyping. Assessing these potential impacts requires a systematic way to measure human-like tone in LLM outputs. We introduce HumT and SocioT, metrics for human-like tone and other dimensions of social perceptions in text data based on relative probabilities from an LLM. By measuring HumT across preference and usage datasets, we find that users prefer less human-like outputs from LLMs in many contexts. HumT also offers insights into the perceptions and impacts of anthropomorphism: human-like LLM outputs are highly correlated with warmth, social closeness, femininity, and low status, which are closely linked to the aforementioned harms. We introduce DumT, a method using HumT to systematically control and reduce the degree of human-like tone while preserving model performance. DumT offers a practical approach for mitigating risks associated with anthropomorphic language generation.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 33 of its 93 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

33 of 93 references.

References · 33

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  7. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  8. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  9. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  10. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  11. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  12. 2025Improving Video Generation with Human Feedbackcited by 267
21 more
  1. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  2. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  3. 2024"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trustcited by 67
  4. 2023SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Modelscited by 66
  5. 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 61
  6. 2024Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Modelscited by 36
  7. 2024Participation in the age of foundation modelscited by 32
  8. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 28
  9. 2024AnthroScore: A Computational Linguistic Measure of Anthropomorphismcited by 27
  10. 2023The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answeringcited by 26
  11. 2020Quantifying Intimacy in Languagecited by 25
  12. 2021Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Modelcited by 23
  13. 2019Unsupervised Discovery of Gendered Language through Latent-Variable Modelingcited by 19
  14. 2024From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust?cited by 18
  15. 2024First-Person Fairness in Chatbotscited by 18
  16. 2024Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliancecited by 15
  17. 2022Mimetic Models: Ethical Implications of AI that Acts Like Youcited by 11
  18. 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
  19. 2021Social Norm Bias: Residual Harms of Fairness-Aware Algorithmscited by 6
  20. 2024Understanding Communication Preferences of Information Workers in Engagement with Text-Based Conversational Agentscited by 6
  21. 2024From Pixels to Personas: Investigating and Modeling Self-Anthropomorphism in Human-Robot Dialoguescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025Training language models to be warm and empathetic makes them less reliable and more sycophanticcited by 19
  2. 2025Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Designcited by 11
  3. 2026OdysSim: Building Foundation Models for Human Behavior Simulationcited by 8
  4. 2025EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferencescited by 3
  5. 2026Learning User Simulators with Turing Rewardscited by 3
  6. 2025Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responsescited by 0
  7. 2026The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Modelscited by 0
  8. 2026Implicit Humanization in Everyday LLM Moral Judgmentscited by 0
  9. 2026Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maximscited by 0
  10. 2026Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Promptscited by 0
  11. 2026Are Human-Aligned Models Models of Humans? A Turing-Test Gap in Preference Alignmentcited by 0