# MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents

arXiv: https://arxiv.org/abs/2602.13372 · HTML: https://mlsanity.com/paper/2602.13372/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 107 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2310.12567/) (2023)
- [OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research](https://mlsanity.com/paper/2305.09304/) (2023)
- [Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation](https://mlsanity.com/paper/2502.06559/) (2025)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Open Problems in Machine Unlearning for AI Safety](https://mlsanity.com/paper/2501.04952/) (2025)
- [Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation](https://mlsanity.com/paper/2405.01677/) (2024)
- [Building Ethics into Artificial Intelligence](https://mlsanity.com/paper/1812.02953/) (2018)
- [Introduction to AI Safety, Ethics, and Society](https://mlsanity.com/paper/2411.01042/) (2024)
- [Thresholded Lexicographic Ordered Multiobjective Reinforcement Learning](https://mlsanity.com/paper/2408.13493/) (2024)
- [Safety through feedback in Constrained RL](https://mlsanity.com/paper/2406.19626/) (2024)
- [Logic Programming and Machine Ethics](https://mlsanity.com/paper/2009.11186/) (2020)
- [Reinforcement Learning and Machine ethics:a systematic review](https://mlsanity.com/paper/2407.02425/) (2024)

## Built on by (most cited first, 2 of 2)

- [AISafetyBenchExplorer: A Metric-Aware Catalogue of AI Safety Benchmarks Reveals Fragmented Measurement and Weak Benchmark Governance](https://mlsanity.com/paper/2604.12875/) (2026)
- [CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models](https://mlsanity.com/paper/2608.08061/) (2026)

## Abstract (arXiv)

Evaluating moral alignment in agents navigating conflicting, hierarchically structured human norms is a critical challenge at the intersection of AI safety, moral philosophy, and cognitive science. We introduce Morality Chains, a novel formalism for representing moral norms as ordered deontic constraints, and MoralityGym, a benchmark of 98 ethical-dilemma problems presented as trolley-dilemma-style Gymnasium environments. By decoupling task-solving from moral evaluation and introducing a novel Morality Metric, MoralityGym allows the integration of insights from psychology and philosophy into the evaluation of norm-sensitive reasoning. Baseline results with Safe RL methods reveal key limitations, underscoring the need for more principled approaches to ethical decision-making. This work provides a foundation for developing AI systems that behave more reliably, transparently, and ethically in complex real-world contexts.