# Position: Stop Making Unscientific AGI Performance Claims

arXiv: https://arxiv.org/abs/2402.03962 · HTML: https://mlsanity.com/paper/2402.03962/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 53 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Talking About Large Language Models](https://mlsanity.com/paper/2212.03551/) (2022)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Causal Parrots: Large Language Models May Talk Causality But Are Not Causal](https://mlsanity.com/paper/2308.13067/) (2023)
- [Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis](https://mlsanity.com/paper/2305.07972/) (2023)
- [Experiments with Random Projection](https://mlsanity.com/paper/1301.3849/) (2013)
- ["AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI](https://mlsanity.com/paper/2309.16606/) (2023)
- [Regularization of Inverse Problems by Neural Networks](https://mlsanity.com/paper/2006.03972/) (2020)
- [Position: AI/ML Influencers Have a Place in the Academic Process](https://mlsanity.com/paper/2401.13782/) (2024)
- [Treat societally impactful scientific insights as open-source software artifacts](https://mlsanity.com/paper/2302.05669/) (2023)

## Built on by (most cited first, 2 of 2)

- [Is In-Context Learning Learning?](https://mlsanity.com/paper/2509.10414/) (2025)
- [Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimization](https://mlsanity.com/paper/2503.07660/) (2025)

## Abstract (arXiv)

Developments in the field of Artificial Intelligence (AI), and particularly large language models (LLMs), have created a 'perfect storm' for observing 'sparks' of Artificial General Intelligence (AGI) that are spurious. Like simpler models, LLMs distill meaningful representations in their latent embeddings that have been shown to correlate with external variables. Nonetheless, the correlation of such representations has often been linked to human-like intelligence in the latter but not the former. We probe models of varying complexity including random projections, matrix decompositions, deep autoencoders and transformers: all of them successfully distill information that can be used to predict latent or external variables and yet none of them have previously been linked to AGI. We argue and empirically demonstrate that the finding of meaningful patterns in latent spaces of models cannot be seen as evidence in favor of AGI. Additionally, we review literature from the social sciences that shows that humans are prone to seek such patterns and anthropomorphize. We conclude that both the methodological setup and common public image of AI are ideal for the misinterpretation that correlations between model representations and some variables of interest are 'caused' by the model's understanding of underlying 'ground truth' relationships. We, therefore, call for the academic community to exercise extra caution, and to be keenly aware of principles of academic integrity, in interpreting and communicating about AI research outcomes.