mlsanityData to 14 Oct 2026

Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model

arXiv 2608.18768 ↗First posted 19 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are widely used to simulate survey respondents, yet their outputs are homogeneous and unfaithful to real inter-group differences, and whether this reflects what a model knows or uses has remained untested. Using representational similarity analysis against Pew American Trends Panel ground truth, we score demographic read-out locations in Mistral-7B and intervene causally across six attribute types. The internal geometry is faithful: attention-head read-outs dominate the standard residual read-out, reaching selection-corrected ρ up to 0.63 -- about 70% of the measurement-reliability ceiling -- and one head, L11 H16, is significantly faithful across all six types, though race-based types stay weak and prompt-fragile, replicating in a second model family. Yet causal use does not track fidelity: the clearest causal pathway (p=0.002) sits in one of the least faithful types, the most faithful type shows no correction-surviving effect, and full identity swaps in the prompt move predictions by under 2% of their error. A 128-dimensional probe on that head lands 21-31% closer to survey truth than the model's answers, yet recovers almost none of the per-question group ordering. Readable, faithfully arranged, and causally used are three dissociable properties of the same model; treating them as one claim is what keeps the "can LLMs simulate populations" debate unresolved.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 19 Aug 2026.

Builds on

32 of 43 references.

References · 32

  1. 2023Mistral 7Bcited by 3,164
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  5. 2023Whose Opinions Do Language Models Reflect?cited by 773
  6. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  7. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  8. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 264
  9. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 255
  10. 2024How to use and interpret activation patchingcited by 214
  11. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  12. 2024Quantifying the Persona Effect in LLM Simulationscited by 169
20 more
  1. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  2. 2023Questioning the Survey Responses of Large Language Modelscited by 74
  3. 2024"My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Modelscited by 69
  4. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 66
  5. 2026Mind the Sim2Real Gap in User Simulation for Agentic Taskscited by 54
  6. 2025Linear Representations of Political Perspective Emerge in Large Language Modelscited by 32
  7. 2025Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamicscited by 19
  8. 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 18
  9. 2026OdysSim: Building Foundation Models for Human Behavior Simulationcited by 13
  10. 2024How Reliable are Causal Probing Interventions?cited by 11
  11. 2025Dissecting Persona-Driven Reasoning in Language Models via Activation Patchingcited by 5
  12. 2026Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMscited by 4
  13. 2026Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Biascited by 4
  14. 2025Localizing Persona Representations in LLMscited by 3
  15. 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
  16. 2026From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMscited by 2
  17. 2026Parametric Social Identity Injection and Diversification in Public Opinion Simulationcited by 2
  18. 2026The Amplifying Mirror: Locating and Steering the Partisan Direction inside a Large Language Modelcited by 2
  19. 2026How Well Do Large Language Models Capture Human Personality?cited by 2
  20. 2026Geometry of Human Perceptual Domains Emerges Transiently in LLM Representationscited by 1

Built on by

Not cited yet.