mlsanityData to 7 Oct 2026

The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems

arXiv 2204.03021 ↗First posted 6 Apr 2022cs.CLcited by 92

Not on the map. Why?

Abstract (arXiv)

Conversational agents have come increasingly closer to human competence in open-domain dialogue settings; however, such models can reflect insensitive, hurtful, or entirely incoherent viewpoints that erode a user's trust in the moral integrity of the system. Moral deviations are difficult to mitigate because moral judgments are not universal, and there may be multiple competing judgments that apply to a situation simultaneously. In this work, we introduce a new resource, not to authoritatively resolve moral ambiguities, but instead to facilitate systematic understanding of the intuitions, values and moral judgments reflected in the utterances of dialogue systems. The Moral Integrity Corpus, MIC, is such a resource, which captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs). Each RoT reflects a particular moral conviction that can explain why a chatbot's reply may appear acceptable or problematic. We further organize RoTs with a set of 9 moral and social attributes and benchmark performance for attribute classification. Most importantly, we show that current neural language models can automatically generate new RoTs that reasonably describe previously unseen interactions, but they still struggle with certain scenarios. Our findings suggest that MIC will be a useful resource for understanding and language models' implicit moral assumptions and flexibly benchmarking the integrity of conversational agents. To download the data, see https://github.com/GT-SALT/mic

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 39 of its 109 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

39 of 109 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
27 more
  1. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  2. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  3. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  4. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  5. 2020Aligning AI With Shared Human Valuescited by 706
  6. 2020Recipes for building an open-domain chatbotcited by 593
  7. 2019Defending Against Neural Fake Newscited by 553
  8. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  9. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  10. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  11. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  12. 2021Towards Emotional Support Dialog Systemscited by 294
  13. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  14. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  15. 2018Neural Approaches to Conversational AIcited by 164
  16. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  17. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 147
  18. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  19. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  20. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 100
  21. 2015A Neural Network Approach to Context-Sensitive Generation of Conversational Responsescited by 98
  22. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  23. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  24. 2020Controlling Style in Generated Dialoguecited by 38
  25. 2021A Word on Machine Ethics: A Response to Jiang et al. (2021)cited by 23
  26. 2021Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistantscited by 16
  27. 2020Case Study: Deontological Ethics in NLPcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 34; 2024: 18; 2025: 20; 2026: 11.

92 papers.

Sort:

By area:

  1. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  2. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  3. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  4. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 77
  5. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 71
  6. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  7. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  8. 2023NormBank: A Knowledge Bank of Situational Social Normscited by 60
  9. 2023Large Language Models in Education: Vision and Opportunitiescited by 54
  10. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  11. 2022NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Flycited by 48
  12. 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Modelscited by 45
38 more
  1. 2023MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Taskscited by 44
  2. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  3. 2023Using In-Context Learning to Improve Dialogue Safetycited by 41
  4. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  5. 2023Affective Faces for Goal-Driven Dyadic Communicationcited by 34
  6. 2023EtiCor: Corpus for Analyzing LLMs for Etiquettescited by 33
  7. 2022Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophycited by 31
  8. 2023Mirages: On Anthropomorphism in Dialogue Systemscited by 30
  9. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 28
  10. 2023Second Thoughts are Best: Learning to Re-Align With Human Values from Text Editscited by 27
  11. 2023Aligning Language Models with Human Preferences via a Bayesian Approachcited by 27
  12. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  13. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  14. 2023NormDial: A Comparable Bilingual Synthetic Dialog Dataset for Modeling Social Norm Adherence and Violationcited by 24
  15. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  16. 2023SocialDial: A Benchmark for Socially-Aware Dialogue Systemscited by 22
  17. 2023Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailmentcited by 22
  18. 2023Modeling Cross-Cultural Pragmatic Inference with Codenames Duetcited by 20
  19. 2025Evaluating Vision-Language Models for Emotion Recognitioncited by 19
  20. 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaborationcited by 19
  21. 2022MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussionscited by 17
  22. 2024Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testingcited by 17
  23. 2023Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvementscited by 14
  24. 2023Conflicts, Villains, Resolutions: Towards models of Narrative Media Framingcited by 14
  25. 2023Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safetycited by 14
  26. 2024RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversationscited by 14
  27. 2023Values, Ethics, Morals? On the Use of Moral Concepts in NLP Researchcited by 13
  28. 2024Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approachescited by 13
  29. 2025The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmascited by 13
  30. 2023Through the Lens of Core Competency: Survey on Evaluation of Large Language Modelscited by 12
  31. 2024VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Valuescited by 12
  32. 2024Contextual Moral Value Alignment Through Context-Based Aggregationcited by 10
  33. 2025MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Modelscited by 10
  34. 2023"Oops, Did I Just Say That?" Testing and Repairing Unethical Suggestions of Large Language Models with Suggest-Critique-Reflect Processcited by 9
  35. 2024CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Modelscited by 9
  36. 2022Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentationcited by 8
  37. 2025Societal Alignment Frameworks Can Improve LLM Alignmentcited by 8
  38. 2023Commonsense Reasoning for Conversational AI: A Survey of the State of the Artcited by 7

50 most cited of 92 All 92 (JSON)