mlsanityData to 7 Oct 2026

Mitigating Covertly Unsafe Text within Natural Language Systems

arXiv 2210.09306 ↗First posted 17 Oct 2022cs.AI cs.CL cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences. However, the degree of explicitness of a generated statement that can cause physical harm varies. In this paper, we distinguish types of text that can lead to physical harm and establish one particularly underexplored category: covertly unsafe text. Then, we further break down this category with respect to the system's information and discuss solutions to mitigate the generation of text in each of these subcategories. Ultimately, our work defines the problem of covertly unsafe language that causes physical harm and argues that this subtle yet dangerous issue needs to be prioritized by stakeholders and regulators. We highlight mitigation strategies to inspire future researchers to tackle this challenging problem and help improve safety within smart systems.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 43 of its 86 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

43 of 86 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  10. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  12. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
31 more
  1. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  2. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  3. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  4. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  5. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  6. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  7. 2018On the Robustness of Interpretability Methodscited by 310
  8. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  9. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  10. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  11. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  12. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  13. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  14. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 146
  15. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  16. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  17. 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
  18. 2017Understanding Abuse: A Typology of Abusive Language Detection Subtaskscited by 115
  19. 2019ASER: A Large-scale Eventuality Knowledge Graphcited by 89
  20. 2019A Just and Comprehensive Strategy for Using NLP to Address Online Abusecited by 49
  21. 2020Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspectivecited by 43
  22. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  23. 2020Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generationcited by 26
  24. 2017Flexible End-to-End Dialogue System for Knowledge Grounded Conversationcited by 18
  25. 2021Investigating Memorization of Conspiracy Theories in Text Generationcited by 15
  26. 2019Detecting Cyberbullying and Cyberaggression in Social Mediacited by 11
  27. 2020A Formal Approach to Explainabilitycited by 9
  28. 2021Open-Domain Question-Answering for COVID-19 and Other Emergent Domainscited by 7
  29. 2018Detecting Gang-Involved Escalation on Social Media Using Contextcited by 5
  30. 2016Using Natural Language Processing and Qualitative Analysis to Intervene in Gang Violence: A Collaboration Between Social Work Researchers and Data Scientistscited by 2
  31. 2017Domain Aware Neural Dialog Systemcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 5; 2025: 1; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  2. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 43
  3. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  4. 2024Near to Mid-term Risks and Opportunities of Open-Source Generative AIcited by 13
  5. 2023ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Modelscited by 12
  6. 2024Risks and Opportunities of Open-Source Generative AIcited by 11
  7. 2024Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitationscited by 10
  8. 2022Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AIcited by 5
  9. 2023Users are the North Star for AI Transparencycited by 5
  10. 2024Unlearning Climate Misinformation in Large Language Modelscited by 4
  11. 2024Purple-teaming LLMs with Adversarial Defender Trainingcited by 4
  12. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 4
1 more
  1. 2026AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialoguecited by 0