mlsanityData to 7 Oct 2026

Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning

arXiv 2303.05670 ↗First posted 10 Mar 2023cs.CL cs.AI cs.CYcited by 7

Not on the map. Why?

Abstract (arXiv)

Due to their similarity-based learning objectives, pretrained sentence encoders often internalize stereotypical assumptions that reflect the social biases that exist within their training corpora. In this paper, we describe several kinds of stereotypes concerning different communities that are present in popular sentence representation models, including pretrained next sentence prediction and contrastive sentence representation models. We compare such models to textual entailment models that learn language logic for a variety of downstream language understanding tasks. By comparing strong pretrained models based on text similarity with textual entailment learning, we conclude that the explicit logic learning with textual entailment can significantly reduce bias and improve the recognition of social communities, without an explicit de-biasing process

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 39 of its 49 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

39 of 49 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  12. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
27 more
  1. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  5. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  6. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  7. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  8. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  9. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  10. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  11. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  12. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  13. 2018Gender Bias in Neural Natural Language Processingcited by 210
  14. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  15. 2020Towards Debiasing Sentence Representationscited by 170
  16. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  17. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  18. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
  19. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  20. 2018Automated Fact Checking: Task formulations, methods and future directionscited by 96
  21. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  22. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  23. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  24. 2019Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniquescited by 65
  25. 2018Gender Aware Spoken Language Translation Applied to English-Arabiccited by 18
  26. 2022Towards Unified Prompt Tuning for Few-shot Text Classificationcited by 18
  27. 2019GeBioToolkit: Automatic Extraction of Gender-Balanced Multilingual Corpus of Wikipedia Biographiescited by 11

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 1; 2025: 1.

7 papers.

  1. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  2. 2023SAIL: Search-Augmented Instruction Learningcited by 33
  3. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  4. 2023Interpretable Unified Language Checkingcited by 26
  5. 2023Holistic Survey of Privacy and Fairness in Machine Learningcited by 4
  6. 2023Entailment as Robust Self-Learnercited by 2
  7. 2025Textual Entailment is not a Better Bias Metric than Token Probabilitycited by 0