mlsanityData to 7 Oct 2026

Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test

arXiv 2402.02135 ↗First posted 3 Feb 2024cs.CL cs.AIcited by 9

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This paper explores the moral judgment and moral reasoning abilities exhibited by Large Language Models (LLMs) across languages through the Defining Issues Test. It is a well known fact that moral judgment depends on the language in which the question is asked. We extend the work of beyond English, to 5 new languages (Chinese, Hindi, Russian, Spanish and Swahili), and probe three LLMs -- ChatGPT, GPT-4 and Llama2Chat-70B -- that shows substantial multilingual text processing and generation abilities. Our study shows that the moral reasoning ability for all models, as indicated by the post-conventional score, is substantially inferior for Hindi and Swahili, compared to Spanish, Russian, Chinese and English, while there is no clear trend for the performance of the latter four languages. The moral judgments too vary considerably by the language.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 20 of its 51 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

20 of 51 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Probing the Moral Development of Large Language Models through Defining Issues Test23 Sep
  2. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?29 Aug
  3. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting11 May
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity8 Feb

Other references · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  6. 2020Aligning AI With Shared Human Valuescited by 706
  7. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  8. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  9. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  10. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  11. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  12. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64
4 more
  1. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
  2. 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Modelscited by 45
  3. 2019MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Predictioncited by 31
  4. 2022The Moral Debater: A Study on the Computational Generation of Morally Framed Argumentscited by 22

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 8.

9 papers.

  1. 2025A Survey of Multilingual Reasoning in Language Modelscited by 31
  2. 2025Evaluating Vision-Language Models for Emotion Recognitioncited by 19
  3. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  4. 2025The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmascited by 12
  5. 2024Evaluating Moral Beliefs across LLMs through a Pluralistic Frameworkcited by 11
  6. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineeringcited by 7
  7. 2025Made-in China, Thinking in America:U.S. Values Persist in Chinese LLMscited by 5
  8. 2025RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivitycited by 4
  9. 2025Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in
  2. 2024Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment
  3. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning
  4. 2024Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models
  5. 2026Untangling Input Language from Reasoning Language: A Diagnostic Framework for Cross-Lingual Moral Alignment in LLMs
  6. 2025Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
  7. 2024MoralBench: Moral Evaluation of LLMs
  8. 2023Exploring the psychology of LLMs' Moral and Legal Reasoning