mlsanityData to 14 Oct 2026

Moral Persuasion in Large Language Models: Evaluating Susceptibility and Ethical Alignment

arXiv 2411.11731 ↗First posted 18 Nov 2024cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

We explore how large language models (LLMs) can be influenced by prompting them to alter their initial decisions and align them with established ethical frameworks. Our study is based on two experiments designed to assess the susceptibility of LLMs to moral persuasion. In the first experiment, we examine the susceptibility to moral ambiguity by evaluating a Base Agent LLM on morally ambiguous scenarios and observing how a Persuader Agent attempts to modify the Base Agent's initial decisions. The second experiment evaluates the susceptibility of LLMs to align with predefined ethical frameworks by prompting them to adopt specific value alignments rooted in established philosophical theories. The results demonstrate that LLMs can indeed be persuaded in morally charged scenarios, with the success of persuasion depending on factors such as the model used, the complexity of the scenario, and the conversation length. Notably, LLMs of distinct sizes but from the same company produced markedly different outcomes, highlighting the variability in their susceptibility to ethical persuasion.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

15 of 21 references.

References · 15

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  3. 2020Aligning AI With Shared Human Valuescited by 723
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  5. 2023An Overview of Catastrophic AI Riskscited by 222
  6. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  7. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
  8. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  9. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  10. 2023Moral Foundations of Large Language Modelscited by 85
  11. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 74
  12. 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 58
3 more
  1. 2024MoralBench: Moral Evaluation of LLMscited by 50
  2. 2023How susceptible are LLMs to Logical Fallacies?cited by 23
  3. 2024An Assessment of Model-On-Model Deceptioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses
  2. 2025"Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas
  3. 2026The Fragility Of Moral Judgment In Large Language Models
  4. 2025Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
  5. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmas
  6. 2025Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment
  7. 2024Exploring and steering the moral compass of Large Language Models
  8. 2025LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models