# Susceptibility to Influence of Large Language Models

arXiv: https://arxiv.org/abs/2303.06074 · HTML: https://mlsanity.com/paper/2303.06074/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations](https://mlsanity.com/paper/2301.04246/) (2023)
- [Who is GPT-3? An Exploration of Personality, Values and Demographics](https://mlsanity.com/paper/2209.14338/) (2022)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [Putting GPT-3's Creativity to the (Alternative Uses) Test](https://mlsanity.com/paper/2206.08932/) (2022)
- [MAUVE Scores for Generative Models: Theory and Practice](https://mlsanity.com/paper/2212.14578/) (2022)
- [Video Generative Adversarial Networks: A Review](https://mlsanity.com/paper/2011.02250/) (2020)

## Built on by (most cited first, 17 of 17)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities](https://mlsanity.com/paper/2308.12833/) (2023)
- [Do LLM Agents Exhibit Social Behavior?](https://mlsanity.com/paper/2312.15198/) (2023)
- [Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information](https://mlsanity.com/paper/2402.18144/) (2024)
- [Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications](https://mlsanity.com/paper/2411.06837/) (2024)
- [AI for social science and social science of AI: A Survey](https://mlsanity.com/paper/2401.11839/) (2024)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)
- [Stick to your Role! Stability of Personal Values Expressed in Large Language Models](https://mlsanity.com/paper/2402.14846/) (2024)
- [Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models](https://mlsanity.com/paper/2306.03799/) (2023)
- [SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabia](https://mlsanity.com/paper/2503.17485/) (2025)
- [Metacognitive Myopia in Large Language Models](https://mlsanity.com/paper/2408.05568/) (2024)
- [A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities](https://mlsanity.com/paper/2507.08425/) (2025)
- [Social Simulations with Large Language Model Risk Utopian Illusion](https://mlsanity.com/paper/2510.21180/) (2025)
- [What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates](https://mlsanity.com/paper/2607.02507/) (2026)
- [Online Anti-sexist Speech: Identifying Resistance to Gender Bias in Political Discourse](https://mlsanity.com/paper/2508.11434/) (2025)

## Abstract (arXiv)

Two studies tested the hypothesis that a Large Language Model (LLM) can be used to model psychological change following exposure to influential input. The first study tested a generic mode of influence - the Illusory Truth Effect (ITE) - where earlier exposure to a statement (through, for example, rating its interest) boosts a later truthfulness test rating. Data was collected from 1000 human participants using an online experiment, and 1000 simulated participants using engineered prompts and LLM completion. 64 ratings per participant were collected, using all exposure-test combinations of the attributes: truth, interest, sentiment and importance. The results for human participants reconfirmed the ITE, and demonstrated an absence of effect for attributes other than truth, and when the same attribute is used for exposure and test. The same pattern of effects was found for LLM-simulated participants. The second study concerns a specific mode of influence - populist framing of news to increase its persuasion and political mobilization. Data from LLM-simulated participants was collected and compared to previously published data from a 15-country experiment on 7286 human participants. Several effects previously demonstrated from the human study were replicated by the simulated study, including effects that surprised the authors of the human study by contradicting their theoretical expectations (anti-immigrant framing of news decreases its persuasion and mobilization); but some significant relationships found in human data (modulation of the effectiveness of populist framing according to relative deprivation of the participant) were not present in the LLM data. Together the two studies support the view that LLMs have potential to act as models of the effect of influence.