# Preventing Another Tessa: Modular Safety Middleware For Health-Adjacent AI Assistants

arXiv: https://arxiv.org/abs/2509.07022 · HTML: https://mlsanity.com/paper/2509.07022/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 12 references that are in the map.
- First posted on arXiv 7 Sep 2025.

## Builds on (most cited first, 4 of 4)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)

## Built on by (most cited first, 3 of 3)

- [Compartmentalised Agentic Reasoning for Clinical NLI](https://mlsanity.com/paper/2509.10222/) (2025)
- [Theory Trace Card: Theory-Driven Socio-Cognitive Evaluation of LLMs](https://mlsanity.com/paper/2601.01878/) (2026)
- [Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study](https://mlsanity.com/paper/2607.21988/) (2026)

## Abstract (arXiv)

In 2023, the National Eating Disorders Association's (NEDA) chatbot Tessa was suspended after providing harmful weight-loss advice to vulnerable users-an avoidable failure that underscores the risks of unsafe AI in healthcare contexts. This paper examines Tessa as a case study in absent safety engineering and demonstrates how a lightweight, modular safeguard could have prevented the incident. We propose a hybrid safety middleware that combines deterministic lexical gates with an in-line large language model (LLM) policy filter, enforcing fail-closed verdicts and escalation pathways within a single model call. Using synthetic evaluations, we show that this design achieves perfect interception of unsafe prompts at baseline cost and latency, outperforming traditional multi-stage pipelines. Beyond technical remedies, we map Tessa's failure patterns to established frameworks (OWASP LLM Top10, NIST SP 800-53), connecting practical safeguards to actionable governance controls. The results highlight that robust, auditable safety in health-adjacent AI does not require heavyweight infrastructure: explicit, testable checks at the last mile are sufficient to prevent "another Tessa", while governance and escalation ensure sustainability in real-world deployment.