# Large Language Models and the Reverse Turing Test

arXiv: https://arxiv.org/abs/2207.14382 · HTML: https://mlsanity.com/paper/2207.14382/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 84 references that are in the map.
- First posted on arXiv 28 Jul 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [From Motor Control to Team Play in Simulated Humanoid Football](https://mlsanity.com/paper/2105.12196/) (2021)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence](https://mlsanity.com/paper/2002.04806/) (2020)
- [Brain-inspired computing: We need a master plan](https://mlsanity.com/paper/2104.14517/) (2021)
- [Review of The Book of Why: The New Science of Cause and Effect](https://mlsanity.com/paper/2003.11635/) (2020)
- [Diversity-enabled sweet spots in layered architectures and speed-accuracy trade-offs in sensorimotor control](https://mlsanity.com/paper/1909.08601/) (2019)
- [Internal Feedback in Biological Control: Locality and System Level Synthesis](https://mlsanity.com/paper/2109.11757/) (2021)

## Built on by (most cited first, 30 of 38)

- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution](https://mlsanity.com/paper/2210.08340/) (2022)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding](https://mlsanity.com/paper/2310.19671/) (2023)
- [Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception](https://mlsanity.com/paper/2310.13712/) (2023)
- [How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO](https://mlsanity.com/paper/2404.13957/) (2024)
- [Attributions toward Artificial Agents in a modified Moral Turing Test](https://mlsanity.com/paper/2406.11854/) (2024)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [LLM-driven Imitation of Subrational Behavior : Illusion or Reality?](https://mlsanity.com/paper/2402.08755/) (2024)
- [Observations on LLMs for Telecom Domain: Capabilities and Limitations](https://mlsanity.com/paper/2305.13102/) (2023)
- [IMPersona: Evaluating Individual Level LM Impersonation](https://mlsanity.com/paper/2504.04332/) (2025)
- [Transformers and Cortical Waves: Encoders for Pulling In Context Across Time](https://mlsanity.com/paper/2401.14267/) (2024)
- [AI Awareness](https://mlsanity.com/paper/2504.20084/) (2025)
- [Eight challenges in developing theory of intelligence](https://mlsanity.com/paper/2306.11232/) (2023)
- [Aligning Large Language Models for Clinical Tasks](https://mlsanity.com/paper/2309.02884/) (2023)
- [Can Large Language Model Summarizers Adapt to Diverse Scientific Communication Goals?](https://mlsanity.com/paper/2401.10415/) (2024)
- [Digital Player: Evaluating Large Language Models based Human-like Agent in Games](https://mlsanity.com/paper/2502.20807/) (2025)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence](https://mlsanity.com/paper/2411.07533/) (2024)
- [Survey of Consciousness Theory from Computational Perspective](https://mlsanity.com/paper/2309.10063/) (2023)
- [Bio-inspired AI: Integrating Biological Complexity into Artificial Intelligence](https://mlsanity.com/paper/2411.15243/) (2024)
- [Meta predictive learning model of languages in neural circuits](https://mlsanity.com/paper/2309.04106/) (2023)
- [Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems: Language Acquisition as a Core for Future Metrics](https://mlsanity.com/paper/2309.11981/) (2023)
- [LLMs grasp morality in concept](https://mlsanity.com/paper/2311.02294/) (2023)
- [MindSemantix: Deciphering Brain Visual Experiences with a Brain-Language Model](https://mlsanity.com/paper/2405.18812/) (2024)
- [A Perspective on Large Language Models, Intelligent Machines, and Knowledge Acquisition](https://mlsanity.com/paper/2408.06598/) (2024)
- [When Autonomy Breaks: The Hidden Existential Risk of AI](https://mlsanity.com/paper/2503.22151/) (2025)
- [Inverse Knowledge Search over Verifiable Reasoning: Synthesizing a Scientific Encyclopedia from a Long Chains-of-Thought Knowledge Base](https://mlsanity.com/paper/2510.26854/) (2025)
- [General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance](https://mlsanity.com/paper/2307.14283/) (2023)
- [From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models](https://mlsanity.com/paper/2312.17256/) (2023)

## Abstract (arXiv)

Large Language Models (LLMs) have been transformative. They are pre-trained foundational models that are self-supervised and can be adapted with fine tuning to a wide range of natural language tasks, each of which previously would have required a separate network model. This is one step closer to the extraordinary versatility of human language. GPT-3 and more recently LaMDA can carry on dialogs with humans on many topics after minimal priming with a few examples. However, there has been a wide range of reactions and debate on whether these LLMs understand what they are saying or exhibit signs of intelligence. This high variance is exhibited in three interviews with LLMs reaching wildly different conclusions. A new possibility was uncovered that could explain this divergence. What appears to be intelligence in LLMs may in fact be a mirror that reflects the intelligence of the interviewer, a remarkable twist that could be considered a Reverse Turing Test. If so, then by studying interviews we may be learning more about the intelligence and beliefs of the interviewer than the intelligence of the LLMs. As LLMs become more capable they may transform the way we interact with machines and how they interact with each other. Increasingly, LLMs are being coupled with sensorimotor devices. LLMs can talk the talk, but can they walk the walk? A road map for achieving artificial general autonomy is outlined with seven major improvements inspired by brain systems. LLMs could be used to uncover new insights into brain function by downloading brain data during natural behaviors.