# Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems: Language Acquisition as a Core for Future Metrics

arXiv: https://arxiv.org/abs/2309.11981 · HTML: https://mlsanity.com/paper/2309.11981/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 23 of its 137 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 23 of 23)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods](https://mlsanity.com/paper/1903.11260/) (2019)
- [Sophisticated Inference](https://mlsanity.com/paper/2006.04120/) (2020)
- [Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods](https://mlsanity.com/paper/1907.09358/) (2019)
- [Large Language Models and the Reverse Turing Test](https://mlsanity.com/paper/2207.14382/) (2022)
- [Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systems](https://mlsanity.com/paper/2109.09904/) (2021)
- [Designing Ecosystems of Intelligence from First Principles](https://mlsanity.com/paper/2212.01354/) (2022)
- [Machine learning on small size samples: A synthetic knowledge synthesis](https://mlsanity.com/paper/2103.01002/) (2021)
- [When we can trust computers (and when we can't)](https://mlsanity.com/paper/2007.03741/) (2020)
- [A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robots](https://mlsanity.com/paper/2103.08183/) (2021)
- [Building Human-like Communicative Intelligence: A Grounded Perspective](https://mlsanity.com/paper/2201.02734/) (2022)
- [Symmetry and Complexity in Object-Centric Deep Active Inference Models](https://mlsanity.com/paper/2304.14493/) (2023)
- [Moving Beyond the Turing Test with the Allen AI Science Challenge](https://mlsanity.com/paper/1604.04315/) (2016)
- [Generalizable Neuro-symbolic Systems for Commonsense Question Answering](https://mlsanity.com/paper/2201.06230/) (2022)
- [Towards A Measure Of General Machine Intelligence](https://mlsanity.com/paper/2109.12075/) (2021)
- [Emergent Linguistic Structures in Neural Networks are Fragile](https://mlsanity.com/paper/2210.17406/) (2022)
- [Can I say, now machines can think?](https://mlsanity.com/paper/2307.07526/) (2023)
- [Intelligence as information processing: brains, swarms, and computers](https://mlsanity.com/paper/2108.05349/) (2021)
- [In the realm of hybrid Brain: Human Brain and AI](https://mlsanity.com/paper/2210.01461/) (2022)

## Built on by (most cited first, 2 of 2)

- [Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation](https://mlsanity.com/paper/2511.01649/) (2025)
- [Lexical Consensus: Grounded Word Learning and Shared Meaning in Artificial Agents](https://mlsanity.com/paper/2606.22207/) (2026)

## Abstract (arXiv)

In the burgeoning field of artificial intelligence (AI), the unprecedented progress of large language models (LLMs) in natural language processing (NLP) offers an opportunity to revisit the entire approach of traditional metrics of machine intelligence, both in form and content. As the realm of machine cognitive evaluation has already reached Imitation, the next step is an efficient Language Acquisition and Understanding. Our paper proposes a paradigm shift from the established Turing Test towards an all-embracing framework that hinges on language acquisition, taking inspiration from the recent advancements in LLMs. The present contribution is deeply tributary of the excellent work from various disciplines, point out the need to keep interdisciplinary bridges open, and delineates a more robust and sustainable approach.