# Black Box Deployed -- Functional Criteria for Artificial Moral Agents in the LLM Era

arXiv: https://arxiv.org/abs/2507.13175 · HTML: https://mlsanity.com/paper/2507.13175/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 59 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 16 of 16)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023)
- [Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in](https://mlsanity.com/paper/2404.18460/) (2024)
- [Moral Alignment for LLM Agents](https://mlsanity.com/paper/2410.01639/) (2024)
- [Language Model Alignment with Elastic Reset](https://mlsanity.com/paper/2312.07551/) (2023)
- [Assessing Political Bias in Large Language Models](https://mlsanity.com/paper/2405.13041/) (2024)
- [Attributions toward Artificial Agents in a modified Moral Turing Test](https://mlsanity.com/paper/2406.11854/) (2024)
- [Minimum Levels of Interpretability for Artificial Moral Agents](https://mlsanity.com/paper/2307.00660/) (2023)
- [PADTHAI-MM: Principles-based Approach for Designing Trustworthy, Human-centered AI using MAST Methodology](https://mlsanity.com/paper/2401.13850/) (2024)

## Abstract (arXiv)

The advancement of powerful yet opaque large language models (LLMs) necessitates a fundamental revision of the philosophical criteria used to evaluate artificial moral agents (AMAs). Pre-LLM frameworks often relied on the assumption of transparent architectures, which LLMs defy due to their stochastic outputs and opaque internal states. This paper argues that traditional ethical criteria are pragmatically obsolete for LLMs due to this mismatch. Engaging with core themes in the philosophy of technology, this paper proffers a revised set of ten functional criteria to evaluate LLM-based artificial moral agents: moral concordance, context sensitivity, normative integrity, metaethical awareness, system resilience, trustworthiness, corrigibility, partial transparency, functional autonomy, and moral imagination. These guideposts, applied to what we term "SMA-LLS" (Simulating Moral Agency through Large Language Systems), aim to steer AMAs toward greater alignment and beneficial societal integration in the coming years. We illustrate these criteria using hypothetical scenarios involving an autonomous public bus (APB) to demonstrate their practical applicability in morally salient contexts.