mlsanityData to 7 Oct 2026

AI-as-exploration: Navigating intelligence space

arXiv 2401.07964 ↗First posted 15 Jan 2024cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence is a field that lives many lives, and the term has come to encompass a motley collection of scientific and commercial endeavours. In this paper, I articulate the contours of a rather neglected but central scientific role that AI has to play, which I dub `AI-as-exploration'.The basic thrust of AI-as-exploration is that of creating and studying systems that can reveal candidate building blocks of intelligence that may differ from the forms of human and animal intelligence we are familiar with. In other words, I suggest that AI is one of the best tools we have for exploring intelligence space, namely the space of possible intelligent systems. I illustrate the value of AI-as-exploration by focusing on a specific case study, i.e., recent work on the capacity to combine novel and invented concepts in humans and Large Language Models. I show that the latter, despite showing human-level accuracy in such a task, probably solve it in ways radically different, but no less relevant to intelligence research, to those hypothesised for humans.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 49 references that are in the map. First posted on arXiv 15 Jan 2024.

Builds on

17 of 49 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2022In-context Learning and Induction Headscited by 930
  7. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  8. 2022Transformers learn in-context by gradient descentcited by 714
  9. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  10. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  11. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  12. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
5 more
  1. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 149
  2. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  3. 2020Word meaning in minds and machinescited by 51
  4. 2023The Vector Grounding Problemcited by 39
  5. 2024A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debatescited by 22

Built on by

Not cited yet.