mlsanityData to 7 Oct 2026

Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions

arXiv 2510.08915 ↗First posted 10 Oct 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce and study artificial impressions--patterns in LLMs' internal representations of prompts that resemble human impressions and stereotypes based on language. We fit linear probes on generated prompts to predict impressions according to the two-dimensional Stereotype Content Model (SCM). Using these probes, we study the relationship between impressions and downstream model behavior as well as prompt features that may inform such impressions. We find that LLMs inconsistently report impressions when prompted, but also that impressions are more consistently linearly decodable from their hidden representations. Additionally, we show that artificial impressions of prompts are predictive of the quality and use of hedging in model responses. We also investigate how particular content, stylistic, and dialectal features in prompts impact LLM impressions.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 73 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

22 of 73 references.

References · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  4. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  5. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  6. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  7. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  8. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  9. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
  10. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  11. 2020Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligencecited by 81
  12. 2024Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discriminationcited by 55
10 more
  1. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  2. 2022VALUE: Understanding Dialect Disparity in NLUcited by 45
  3. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  4. 2022Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Modelscited by 35
  5. 2024Empowering Personalized Learning through a Conversation-based Tutoring System with Student Modelingcited by 33
  6. 2022A Robust Bias Mitigation Procedure Based on the Stereotype Content Modelcited by 24
  7. 2024ReIFE: Re-evaluating Instruction-Following Evaluationcited by 22
  8. 2024NegativePrompt: Leveraging Psychology for Large Language Models Enhancement via Negative Emotional Stimulicited by 18
  9. 2024Re-evaluating Automatic LLM System Ranking for Alignment with Human Preferencecited by 16
  10. 2025Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systemscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Every Token Counts: Exact Likert-Scale Distributions for Measuring LLM Attitudes and Biasescited by 3
  2. 2026The Hrunting of AI: Where and How to Improve English Dialectal Fairnesscited by 0