mlsanityData to 14 Oct 2026

NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps

arXiv 2404.01651 ↗First posted 2 Apr 2024cs.CL cs.CY cs.HCcited by 20

Not on the map. Why?

Abstract (arXiv)

The use of words to convey speaker's intent is traditionally distinguished from the `mention' of words for quoting what someone said, or pointing out properties of a word. Here we show that computationally modeling this use-mention distinction is crucial for dealing with counterspeech online. Counterspeech that refutes problematic content often mentions harmful language but is not harmful itself (e.g., calling a vaccine dangerous is not the same as expressing disapproval of someone for calling vaccines dangerous). We show that even recent language models fail at distinguishing use from mention, and that this failure propagates to two key downstream tasks: misinformation and hate speech detection, resulting in censorship of counterspeech. We introduce prompting mitigations that teach the use-mention distinction, and show they reduce these errors. Our work highlights the importance of the use-mention distinction for NLP and CSS and offers ways to address it.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 34 of its 91 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

34 of 91 references.

References · 34

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  4. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  5. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
  6. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 221
  7. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  8. 2019A Benchmark Dataset for Learning to Intervene in Online Hate Speechcited by 129
  9. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  10. 2019CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speechcited by 110
  11. 2021Challenges in Automated Debiasing for Toxic Language Detectioncited by 96
  12. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 94
22 more
  1. 2021Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speechcited by 82
  2. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 70
  3. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 69
  4. 2018Thou shalt not hate: Countering Online Hate Speechcited by 64
  5. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 61
  6. 2018Challenges for Toxic Comment Classification: An In-Depth Error Analysiscited by 56
  7. 2022Handling and Presenting Harmful Text in NLP Researchcited by 43
  8. 2021Towards Knowledge-Grounded Counter Narrative Generation for Hate Speechcited by 42
  9. 2022Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Counteringcited by 40
  10. 2023Reinforcement Learning-based Counter-Misinformation Response Generation: A Case Study of COVID-19 Vaccine Misinformationcited by 39
  11. 2022Hate Speech and Counter Speech Detection: Conversational Context Does Mattercited by 32
  12. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  13. 2021Defining and Detecting Toxicity on Social Media: Context and Knowledge are Keycited by 29
  14. 2023From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Modelscited by 26
  15. 2023Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Languagecited by 24
  16. 2020Countering hate on social media: Large scale classification of hate and counter speechcited by 22
  17. 2021Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Frameworkcited by 19
  18. 2022Detecting Unintended Social Bias in Toxic Language Datasetscited by 12
  19. 2020Impact and dynamics of hate and counter speech onlinecited by 11
  20. 2023string2string: A Modern Python Library for String-to-String Algorithmscited by 9
  21. 2021Evaluating Neural Word Embeddings for Sanskritcited by 8
  22. 2020Don't Let Me Be Misunderstood: Comparing Intentions and Perceptions in Online Discussionscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 11.

20 papers.

Sort:

By area:

  1. 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 18
  2. 2024HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twittercited by 13
  3. 2025When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Editscited by 3
  4. 2025FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarkingcited by 3
  5. 2024On the Role of Speech Data in Reducing Toxicity Detection Biascited by 2
  6. 2025Revealing Hidden Mechanisms of Cross-Country Content Moderation with Natural Language Processingcited by 2
  7. 2024When in Doubt, Cascade: Towards Building Efficient and Capable Guardrailscited by 1
  8. 2026Speaking of Language: Reflections on Metalanguage Research in NLPcited by 1
  9. 2026The Enforcement and Feasibility of Hate Speech Moderationcited by 1
  10. 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructurecited by 1
  11. 2026Instruction-Tuned Language Models Cannot Sample from Distributions They Can Describecited by 1
  12. 2025Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeechcited by 0
8 more
  1. 2025Are LLMs Good Safety Agents or a Propaganda Engine?cited by 0
  2. 2026LinGO: A Linguistic Graph Optimization Framework with LLMs for Interpreting Intents of Online Uncivil Discoursecited by 0
  3. 2026Probing Cultural Signals in Large Language Models through Author Profilingcited by 0
  4. 2026CHAIRO: Contextual Hierarchical Analogical Induction and Reasoning Optimization for LLMscited by 0
  5. 2026Evaluating Large Language Models for Antisemitic Incident Classificationcited by 0
  6. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  7. 2026Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderationcited by 0
  8. 2026How Do People Challenge Racial Stereotypes Online? Counter-Story Detection Across Reddit Communitiescited by 0