# RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue

arXiv: https://arxiv.org/abs/2606.13310 · HTML: https://mlsanity.com/paper/2606.13310/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems](https://mlsanity.com/paper/2503.03750/) (2025)
- [Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction](https://mlsanity.com/paper/2407.13943/) (2024)
- [The Traitors: Deception and Trust in Multi-Agent Language Model Simulations](https://mlsanity.com/paper/2505.12923/) (2025)
- [Eliciting Secret Knowledge from Language Models](https://mlsanity.com/paper/2510.01070/) (2025)
- [Among Us: A Sandbox for Measuring and Detecting Agentic Deception](https://mlsanity.com/paper/2504.04072/) (2025)
- [BeHonest: Benchmarking Honesty in Large Language Models](https://mlsanity.com/paper/2406.13261/) (2024)
- [When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Models](https://mlsanity.com/paper/2506.04909/) (2025)
- [MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind](https://mlsanity.com/paper/2504.18039/) (2025)
- [When Do Large Language Models Exhibit Unsolicited Deception?](https://mlsanity.com/paper/2504.00285/) (2025)
- [Preference Learning with Lie Detectors can Induce Honesty or Evasion](https://mlsanity.com/paper/2505.13787/) (2025)
- [Bayesian Social Deduction with Graph-Informed Language Models](https://mlsanity.com/paper/2506.17788/) (2025)
- [Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia](https://mlsanity.com/paper/2509.23023/) (2025)
- [Among Them: A game-based framework for assessing persuasion capabilities of LLMs](https://mlsanity.com/paper/2502.20426/) (2025)
- [The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind](https://mlsanity.com/paper/2509.20393/) (2025)
- [Can LLMs Lie? Investigation beyond Hallucination](https://mlsanity.com/paper/2509.03518/) (2025)
- [Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games](https://mlsanity.com/paper/2506.05309/) (2025)
- [OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulation](https://mlsanity.com/paper/2504.13707/) (2025)
- [Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing](https://mlsanity.com/paper/2603.07202/) (2026)

## Built on by (most cited first, 1 of 1)

- [Building Socio-Affective Artificial Intelligence for Interactive Multi-Agent Simulations](https://mlsanity.com/paper/2609.26927/) (2026)

## Abstract (arXiv)

The original Turing Test asks a human judge to distinguish a machine from a person through dialogue. Three quarters of a century later, conversational systems pass this test in casual settings; the interesting epistemological question has shifted. We argue that the relevant modern variant asks not whether a dialogue partner is artificial, but whether it can be trusted. We present RogueAI, an interactive webapp that operationalizes this revisited test as a one-on-two interrogation game: a human player questions two indistinguishable Large Language Model agents, knowing that exactly one of them has been licensed to deceive within a shared fictional scenario. The player's task is to identify the deceptive agent and "shut it off" before a turn budget is exhausted. We further introduce AutoRogueAI, a procedural extension in which players co-design a custom scenario with a narrator agent that secretly chooses its own deception strategy. We describe the framing, sketch the abstract architecture and gameplay loop, and situate the artifact within recent work on LLM deception, social-deduction benchmarks, and scalable oversight via debate. A three-day pilot deployment (467 initiated sessions, 415 completed, 1876 interaction turns in Italian) provides early feasibility evidence and surfaces a concrete tension: the deceptive agent carries a reliable, locally-present linguistic signature - differential helpfulness, brevity, hedging - that a simple heuristic exploits at 75.6% accuracy, yet human players achieved only 56.6%, consistent with ignoring the most diagnostic signal entirely. We discuss what this gap implies for the artifact's use as a data-collection vehicle, a teaching tool, and an evaluation harness for honesty-trained models.