How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
As LLMs take on roles requiring moral advice, understanding how they attribute moral agency becomes critical. Humans possess moral agency, the capacity to make ethically guided decisions and bear responsibility for their consequences, a well-established construct in moral psychology. Yet as artificial agents (AAs) such as robots, drones, and disembodied AI systems become increasingly embedded in smart city environments, the question of whether and how moral agency is attributed to them takes on new urgency. This paper presents, to the best of our knowledge, the first empirical study comparing how humans and LLMs evaluate perceived moral agency (PMA) across human and autonomous artificial agents varying in embodiment, situated in plausible smart city scenarios. Using an adaptation of a validated PMA scale, we applied a protocol to 190 human participants as well as various LLMs. Our evaluation reveals higher perceptions of moral agency in humans than in AAs. However, when facing moral dilemmas in concrete scenarios, LLMs reason outward from the situation, prioritizing harm severity and contextual urgency over any stable assessment of the agent itself, amplifying a context-sensitivity also present in human raters. These findings are particularly relevant as LLMs become increasingly involved in everyday moral decisions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 16 of its 57 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
16 of 57 references.
Closest ancestors · 1
Same direction, newest first
Other references · 15
- 2023Whose Opinions Do Language Models Reflect?cited by 773
- 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
- 2023Personality Traits in Large Language Modelscited by 169
- 2015The social dilemma of autonomous vehiclescited by 48
- 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
- 2024Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidancecited by 32
- 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
- 2024MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussionscited by 23
- 2025LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Modelscited by 21
- 2025Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Frameworkcited by 16
- 2025EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Modelscited by 6
- 2025Do Large Language Models Understand Morality Across Cultures?cited by 5
3 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
- 2025The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis
- 2025Black Box Deployed -- Functional Criteria for Artificial Moral Agents in the LLM Era
- 2026The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers
- 2026Are LLMs Bad at Moral Reasoning?
- 2023Exploring the psychology of LLMs' Moral and Legal Reasoning
- 2023Do LLM Agents Exhibit Social Behavior?
- 2026A three-dimensional typology of agency for advanced AI systems