mlsanityData to 7 Oct 2026

Susceptibility to Influence of Large Language Models

arXiv 2303.06074 ↗First posted 10 Mar 2023cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Two studies tested the hypothesis that a Large Language Model (LLM) can be used to model psychological change following exposure to influential input. The first study tested a generic mode of influence - the Illusory Truth Effect (ITE) - where earlier exposure to a statement (through, for example, rating its interest) boosts a later truthfulness test rating. Data was collected from 1000 human participants using an online experiment, and 1000 simulated participants using engineered prompts and LLM completion. 64 ratings per participant were collected, using all exposure-test combinations of the attributes: truth, interest, sentiment and importance. The results for human participants reconfirmed the ITE, and demonstrated an absence of effect for attributes other than truth, and when the same attribute is used for exposure and test. The same pattern of effects was found for LLM-simulated participants. The second study concerns a specific mode of influence - populist framing of news to increase its persuasion and political mobilization. Data from LLM-simulated participants was collected and compared to previously published data from a 15-country experiment on 7286 human participants. Several effects previously demonstrated from the human study were replicated by the simulated study, including effects that surprised the authors of the human study by contradicting their theoretical expectations (anti-immigrant framing of news decreases its persuasion and mobilization); but some significant relationships found in human data (modulation of the effectiveness of populist framing according to relative deprivation of the participant) were not present in the LLM data. Together the two studies support the view that LLMs have potential to act as models of the effect of influence.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 19 of its 55 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

19 of 55 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  8. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  9. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  10. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  11. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  12. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
7 more
  1. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 157
  2. 2022Who is GPT-3? An Exploration of Personality, Values and Demographicscited by 112
  3. 2022When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgmentcited by 93
  4. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79
  5. 2022Putting GPT-3's Creativity to the (Alternative Uses) Testcited by 62
  6. 2022MAUVE Scores for Generative Models: Theory and Practicecited by 34
  7. 2020Video Generative Adversarial Networks: A Reviewcited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 7; 2025: 4; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  3. 2023Challenges and Applications of Large Language Modelscited by 251
  4. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  5. 2023Do LLM Agents Exhibit Social Behavior?cited by 53
  6. 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 48
  7. 2024Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implicationscited by 30
  8. 2024AI for social science and social science of AI: A Surveycited by 29
  9. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  10. 2024Stick to your Role! Stability of Personal Values Expressed in Large Language Modelscited by 14
  11. 2023Prompt Space Optimizing Few-shot Reasoning Success with Large Language Modelscited by 11
  12. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
5 more
  1. 2024Metacognitive Myopia in Large Language Modelscited by 6
  2. 2025A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunitiescited by 2
  3. 2025Social Simulations with Large Language Model Risk Utopian Illusioncited by 2
  4. 2026What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debatescited by 1
  5. 2025Online Anti-sexist Speech: Identifying Resistance to Gender Bias in Political Discoursecited by 0