Moral Persuasion in Large Language Models: Evaluating Susceptibility and Ethical Alignment
Not on the map. Why?
Abstract (arXiv)
We explore how large language models (LLMs) can be influenced by prompting them to alter their initial decisions and align them with established ethical frameworks. Our study is based on two experiments designed to assess the susceptibility of LLMs to moral persuasion. In the first experiment, we examine the susceptibility to moral ambiguity by evaluating a Base Agent LLM on morally ambiguous scenarios and observing how a Persuader Agent attempts to modify the Base Agent's initial decisions. The second experiment evaluates the susceptibility of LLMs to align with predefined ethical frameworks by prompting them to adopt specific value alignments rooted in established philosophical theories. The results demonstrate that LLMs can indeed be persuaded in morally charged scenarios, with the success of persuasion depending on factors such as the model used, the complexity of the scenario, and the conversation length. Notably, LLMs of distinct sizes but from the same company produced markedly different outcomes, highlighting the variability in their susceptibility to ethical persuasion.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 18 Nov 2024.
Builds on
15 of 21 references.
References · 15
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
- 2020Aligning AI With Shared Human Valuescited by 723
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
- 2023An Overview of Catastrophic AI Riskscited by 222
- 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
- 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
- 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
- 2023Moral Foundations of Large Language Modelscited by 85
- 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 74
- 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 58
3 more
- 2024MoralBench: Moral Evaluation of LLMscited by 50
- 2023How susceptible are LLMs to Logical Fallacies?cited by 23
- 2024An Assessment of Model-On-Model Deceptioncited by 5
Built on by
Citing papers per year: 2025: 3; 2026: 7.
10 papers.
- 2025Accumulating Context Changes the Beliefs of Language Modelscited by 28
- 2025Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PDcited by 13
- 2025Reasoning or Rambling? Exploring the Effect of Thinking on Agent Persuasioncited by 4
- 2026CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulationcited by 2
- 2026Do Emotions Influence Moral Judgment in Large Language Models?cited by 2
- 2026Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMscited by 1
- 2026Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgmentcited by 1
- 2026What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debatescited by 1
- 2026Staying with the Uncertainty: Uncertainty-Scaffolding Strategies for Artificial Moral Advisors in LLM-to-LLM Simulated Conversationscited by 0
- 2026Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cuescited by 0
- 2026What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates2 Jul 2026
- 2026Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues30 Jun 2026
- 2026Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment12 Jun 2026
- 2026Staying with the Uncertainty: Uncertainty-Scaffolding Strategies for Artificial Moral Advisors in LLM-to-LLM Simulated Conversations4 Jun 2026
- 2026Do Emotions Influence Moral Judgment in Large Language Models?21 Apr 2026
- 2026CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulation10 Apr 2026
- 2026Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs10 Mar 2026
- 2025Accumulating Context Changes the Beliefs of Language Models3 Nov 2025
- 2025Reasoning or Rambling? Exploring the Effect of Thinking on Agent Persuasion25 Sep 2025
- 2025Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PD24 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2024Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses
- 2025"Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas
- 2026The Fragility Of Moral Judgment In Large Language Models
- 2025Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
- 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmas
- 2025Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment
- 2024Exploring and steering the moral compass of Large Language Models
- 2025LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models