# RV4Chatbot: Are Chatbots Allowed to Dream of Electric Sheep?

arXiv: https://arxiv.org/abs/2411.14368 · HTML: https://mlsanity.com/paper/2411.14368/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 44 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 7 of 7)

- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Low-Resource Languages Jailbreak GPT-4](https://mlsanity.com/paper/2310.02446/) (2023)
- [Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models](https://mlsanity.com/paper/2310.02949/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Rasa: Open Source Language Understanding and Dialogue Management](https://mlsanity.com/paper/1712.05181/) (2017)
- [The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks](https://mlsanity.com/paper/2310.15469/) (2023)

## Built on by (most cited first, 1 of 1)

- [One Monitor Does Not Fit All: Formalism-Agnostic Runtime Verification for Agentic AI](https://mlsanity.com/paper/2610.19818/) (2026)

## Abstract (arXiv)

Chatbots have become integral to various application domains, including those with safety-critical considerations. As a result, there is a pressing need for methods that ensure chatbots consistently adhere to expected, safe behaviours. In this paper, we introduce RV4Chatbot, a Runtime Verification framework designed to monitor deviations in chatbot behaviour. We formalise expected behaviours as interaction protocols between the user and the chatbot. We present the RV4Chatbot design and describe two implementations that instantiate it: RV4Rasa, for monitoring chatbots created with the Rasa framework, and RV4Dialogflow, for monitoring Dialogflow chatbots. Additionally, we detail experiments conducted in a factory automation scenario using both RV4Rasa and RV4Dialogflow.