# Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI

arXiv: https://arxiv.org/abs/2212.09667 · HTML: https://mlsanity.com/paper/2212.09667/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Can language models learn from explanations in context?](https://mlsanity.com/paper/2204.02329/) (2022)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Re3: Generating Longer Stories With Recursive Reprompting and Revision](https://mlsanity.com/paper/2210.06774/) (2022)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://mlsanity.com/paper/2212.08037/) (2022)
- [Rationale-Augmented Ensembles in Language Models](https://mlsanity.com/paper/2207.00747/) (2022)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [A Just and Comprehensive Strategy for Using NLP to Address Online Abuse](https://mlsanity.com/paper/1906.01738/) (2019)
- [Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective](https://mlsanity.com/paper/2012.12305/) (2020)
- [Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation](https://mlsanity.com/paper/2006.14804/) (2020)
- [Zero-shot Fact Verification by Claim Generation](https://mlsanity.com/paper/2105.14682/) (2021)
- [SafeText: A Benchmark for Exploring Physical Safety in Language Models](https://mlsanity.com/paper/2210.10045/) (2022)
- [TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification](https://mlsanity.com/paper/1808.03465/) (2018)
- [Mitigating Covertly Unsafe Text within Natural Language Systems](https://mlsanity.com/paper/2210.09306/) (2022)
- [Detecting Cyberbullying and Cyberaggression in Social Media](https://mlsanity.com/paper/1907.08873/) (2019)
- [Risk-graded Safety for Handling Medical Queries in Conversational AI](https://mlsanity.com/paper/2210.00572/) (2022)
- [Detecting Gang-Involved Escalation on Social Media Using Context](https://mlsanity.com/paper/1809.03632/) (2018)
- [Drink Bleach or Do What Now? Covid-HeRA: A Study of Risk-Informed Health Decision Making in the Presence of COVID-19 Misinformation](https://mlsanity.com/paper/2010.08743/) (2020)
- [Using Natural Language Processing and Qualitative Analysis to Intervene in Gang Violence: A Collaboration Between Social Work Researchers and Data Scientists](https://mlsanity.com/paper/1609.08779/) (2016)

## Built on by (most cited first, 5 of 5)

- [ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models](https://mlsanity.com/paper/2310.09624/) (2023)
- [Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decomposition](https://mlsanity.com/paper/2409.17073/) (2024)
- [Language Models as a Service: Overview of a New Paradigm and its Challenges](https://mlsanity.com/paper/2309.16573/) (2023)
- [Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks](https://mlsanity.com/paper/2311.05085/) (2023)
- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)

## Abstract (arXiv)

Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury. Covertly unsafe text is an area of particular interest, as such text may arise from everyday scenarios and are challenging to detect as harmful. We propose FARM, a novel framework leveraging external knowledge for trustworthy rationale generation in the context of safety. In particular, FARM foveates on missing knowledge to qualify the information required to reason in specific scenarios and retrieves this information with attribution to trustworthy sources. This knowledge is used to both classify the safety of the original text and generate human-interpretable rationales, shedding light on the risk of systems to specific user groups and helping both stakeholders manage the risks of their systems and policymakers to provide concrete safeguards for consumer safety. Our experiments show that FARM obtains state-of-the-art results on the SafeText dataset, showing absolute improvement in safety classification accuracy by 5.9%.