# Where's the Liability in Harmful AI Speech?

arXiv: https://arxiv.org/abs/2308.04635 · HTML: https://mlsanity.com/paper/2308.04635/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 14 of its 63 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [Ethical Challenges in Data-Driven Dialogue Systems](https://mlsanity.com/paper/1711.09050/) (2017)
- [Posterior calibration and exploratory analysis for natural language processing models](https://mlsanity.com/paper/1508.05154/) (2015)
- [Mutual Information Alleviates Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2210.13210/) (2022)

## Built on by (most cited first, 12 of 12)

- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models](https://mlsanity.com/paper/2401.01301/) (2024)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Immunization against harmful fine-tuning attacks](https://mlsanity.com/paper/2402.16382/) (2024)
- [AI Hallucinations: A Misnomer Worth Clarifying](https://mlsanity.com/paper/2401.06796/) (2024)
- [Towards Trustworthy AI: A Review of Ethical and Robust Large Language Models](https://mlsanity.com/paper/2407.13934/) (2024)
- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [Acceptable Use Policies for Foundation Models](https://mlsanity.com/paper/2409.09041/) (2024)
- [Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation](https://mlsanity.com/paper/2402.01769/) (2024)
- [Sound Check: Auditing Audio Datasets](https://mlsanity.com/paper/2410.13114/) (2024)
- [A Voter-Based Stochastic Rejection-Method Framework for Asymptotically Safe Language Model Outputs](https://mlsanity.com/paper/2407.16994/) (2024)
- [Many-body Tipping Dynamics of ChatGPT-like AIs](https://mlsanity.com/paper/2607.25279/) (2026)

## Abstract (arXiv)

Generative AI, in particular text-based "foundation models" (large models trained on a huge variety of information including the internet), can generate speech that could be problematic under a wide range of liability regimes. Machine learning practitioners regularly "red team" models to identify and mitigate such problematic speech: from "hallucinations" falsely accusing people of serious misconduct to recipes for constructing an atomic bomb. A key question is whether these red-teamed behaviors actually present any liability risk for model creators and deployers under U.S. law, incentivizing investments in safety mechanisms. We examine three liability regimes, tying them to common examples of red-teamed model behaviors: defamation, speech integral to criminal conduct, and wrongful death. We find that any Section 230 immunity analysis or downstream liability analysis is intimately wrapped up in the technical details of algorithm design. And there are many roadblocks to truly finding models (and their associated parties) liable for generated speech. We argue that AI should not be categorically immune from liability in these scenarios and that as courts grapple with the already fine-grained complexities of platform algorithms, the technical details of generative AI loom above with thornier questions. Courts and policymakers should think carefully about what technical design incentives they create as they evaluate these issues.