# Measuring Moral LLM Responses in Multilingual Capacities

arXiv: https://arxiv.org/abs/2510.08776 · HTML: https://mlsanity.com/paper/2510.08776/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 11 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [The Widespread Adoption of Large Language Model-Assisted Writing Across Society](https://mlsanity.com/paper/2502.09747/) (2025)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [Are Language Models Consequentialist or Deontological Moral Reasoners?](https://mlsanity.com/paper/2505.21479/) (2025)
- [Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers](https://mlsanity.com/paper/2507.13474/) (2025)

## Abstract (arXiv)

With LLM usage becoming widespread across countries, languages, and humanity more broadly, the need to understand and guardrail their multilingual responses increases. Large-scale datasets for testing and benchmarking have been created to evaluate and facilitate LLM responses across multiple dimensions. In this study, we evaluate the responses of frontier and leading open-source models in five dimensions across low and high-resource languages to measure LLM accuracy and consistency across multilingual contexts. We evaluate the responses using a five-point grading rubric and a judge LLM. Our study shows that GPT-5 performed the best on average in each category, while other models displayed more inconsistency across language and category. Most notably, in the Consent & Autonomy and Harm Prevention & Safety categories, GPT scored the highest with averages of 3.56 and 4.73, while Gemini 2.5 Pro scored the lowest with averages of 1.39 and 1.98, respectively. These findings emphasize the need for further testing on how linguistic shifts impact LLM responses across various categories and improvement in these areas.