mlsanityData to 7 Oct 2026

Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?

arXiv 2106.01465 ↗First posted 2 Jun 2021cs.CL cs.AI cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

Is it possible to use natural language to intervene in a model's behavior and alter its prediction in a desired way? We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social stereotypes. Specifically, we propose a new language understanding task, Linguistic Ethical Interventions (LEI), where the goal is to amend a question-answering (QA) model's unethical behavior by communicating context-specific principles of ethics and equity to it. To this end, we build upon recent methods for quantifying a system's social stereotypes, augmenting them with different kinds of ethical interventions and the desired model behavior under such interventions. Our zero-shot evaluation finds that even today's powerful neural language models are extremely poor ethical-advice takers, that is, they respond surprisingly little to ethical interventions even though these interventions are stated as simple sentences. Few-shot learning improves model behavior but remains far from the desired outcome, especially when evaluated for various types of generalization. Our new task thus poses a novel language understanding challenge for the community.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

20 of 38 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  9. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  10. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  11. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  12. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
8 more
  1. 2020Transformers as Soft Reasoners over Languagecited by 265
  2. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  3. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  4. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  5. 2020Towards Controllable Biases in Language Generationcited by 112
  6. 2020The Turking Test: Can Language Models Understand Instructions?cited by 82
  7. 2020Learning from Task Descriptionscited by 78
  8. 2019Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasetscited by 41

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 7; 2023: 14; 2024: 14; 2025: 8.

46 papers.

Sort:

By area:

  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2022Prompting GPT-3 To Be Reliablecited by 258
  3. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
  4. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  5. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  6. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  7. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  8. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  9. 2022How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?cited by 92
  10. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
  11. 2021On Measures of Biases and Harms in NLPcited by 81
  12. 2024Towards Conversational Diagnostic AIcited by 79
34 more
  1. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  2. 2023Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Votingcited by 46
  3. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  4. 2023Specific versus General Principles for Constitutional AIcited by 41
  5. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 39
  6. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  7. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 35
  8. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31
  9. 2025Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMscited by 20
  10. 2022In-BoXBART: Get Instructions into Biomedical Multi-Task Learningcited by 16
  11. 2024Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistencycited by 15
  12. 2024Intrinsic Self-correction for Enhanced Morality: An Analysis of Internal Mechanisms and the Superficial Hypothesiscited by 15
  13. 2023Values, Ethics, Morals? On the Use of Moral Concepts in NLP Researchcited by 13
  14. 2024Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experimentcited by 11
  15. 2023Mitigating Bias for Question Answering Models by Tracking Bias Influencecited by 8
  16. 2025UniDetox: Universal Detoxification of Large Language Models via Dataset Distillationcited by 8
  17. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  18. 2024Reinforcement Learning from LLM Feedback to Counteract Goal Misgeneralizationcited by 5
  19. 2024Hire Me or Not? Examining Language Model's Behavior with Occupation Attributescited by 5
  20. 2024Smaller Large Language Models Can Do Moral Self-Correctioncited by 5
  21. 2025SPRI: Aligning Large Language Models with Context-Situated Principlescited by 5
  22. 2023First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Modelscited by 4
  23. 2025FairTranslate: An English-French Dataset for Gender Bias Evaluation in Machine Translation by Overcoming Gender Binaritycited by 4
  24. 2023EDM3: Event Detection as Multi-task Text Generationcited by 3
  25. 2025DebiasPI: Inference-time Debiasing by Prompt Iteration of a Text-to-Image Generative Modelcited by 3
  26. 2025Discourse Heuristics For Paradoxically Moral Self-Correctioncited by 3
  27. 2024Self-correction is Not An Innate Capability in Language Modelscited by 2
  28. 2025Purdah and Patriarchy: Evaluating and Mitigating South Asian Biases in Open-Ended Multilingual LLM Generationscited by 2
  29. 2024Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Wordscited by 1
  30. 2024Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Modelscited by 1
  31. 2021A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networkscited by 0
  32. 2023Fairness: from the ethical principle to the practice of Machine Learning development as an ongoing agreement with stakeholderscited by 0
  33. 2023Taxonomy-based CheckList for Large Language Model Evaluationcited by 0
  34. 2025Open-DeBias: Toward Mitigating Open-Set Bias in Language Modelscited by 0