# Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach

arXiv: https://arxiv.org/abs/2310.08172 · HTML: https://mlsanity.com/paper/2310.08172/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models](https://mlsanity.com/paper/2306.05179/) (2023)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [Mind meets machine: Unravelling GPT-4's cognitive psychology](https://mlsanity.com/paper/2303.11436/) (2023)
- [MoocRadar: A Fine-grained and Multi-aspect Knowledge Repository for Improving Cognitive Student Modeling in MOOCs](https://mlsanity.com/paper/2304.02205/) (2023)

## Built on by (most cited first, 5 of 5)

- [Serial Position Effects of Large Language Models](https://mlsanity.com/paper/2406.15981/) (2024)
- [Taxonomy of User Needs and Actions](https://mlsanity.com/paper/2510.06124/) (2025)
- [Quantifying Cognitive Bias Induction in LLM-Generated Content](https://mlsanity.com/paper/2507.03194/) (2025)
- [Image First or Text First? Optimising the Sequencing of Modalities in Large Language Model Prompting and Reasoning Tasks](https://mlsanity.com/paper/2410.03062/) (2024)
- [Affordably Fine-tuned LLMs Provide Better Answers to Course-specific MCQs](https://mlsanity.com/paper/2501.05891/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have not only exhibited exceptional performance across various tasks, but also demonstrated sparks of intelligence. Recent studies have focused on assessing their capabilities on human exams and revealed their impressive competence in different domains. However, cognitive research on the overall knowledge structure of LLMs is still lacking. In this paper, based on educational diagnostic assessment method, we conduct an evaluation using MoocRadar, a meticulously annotated human test dataset based on Bloom Taxonomy. We aim to reveal the knowledge structures of LLMs and gain insights of their cognitive capabilities. This research emphasizes the significance of investigating LLMs' knowledge and understanding the disparate cognitive patterns of LLMs. By shedding light on models' knowledge, researchers can advance development and utilization of LLMs in a more informed and effective manner.