Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems
Not on the map. Why?
Abstract (arXiv)
Evaluating artificial intelligence systems has historically relied on outcome-based benchmarks that measure task accuracy, robustness, or fairness. While indispensable, these benchmarks provide limited diagnostic insight into the underlying cognitive processes that generate performance-leaving critical questions unanswered about how AI systems reason, integrate memory, manage complexity, or avoid generating false information. This paper introduces the Multi-Dimensional Assessment for AI Cognition (MAAC), a theoretically grounded framework for shifting evaluation from what text-based AI systems produce to how they think. MAAC defines nine cognitively motivated dimensions: Cognitive Load, Tool Execution, Content Quality, Memory Integration, Complexity Handling, Hallucination Control, Knowledge Transfer, Processing Efficiency, and Process-Outcome Alignment. Each dimension is grounded in established cognitive science theory-drawing on Marr's tri-level hypothesis, Baddeley's working memory model, Sweller's cognitive load theory, and unified theories of cognition. Five theoretical analyses provide initial support for the framework's coherence and empirical testability: dimension-to-theory mapping; a coverage matrix assessing breadth and non-redundancy; a formal gap analysis relative to current evaluation practice; a worked diagnostic illustration; and a set of a priori interdependency predictions for future empirical testing. MAAC provides a theoretical and operational framework for principled process-level cognitive assessment of text-based AI systems, complementing existing outcome-based benchmarks with cognitively grounded, multi-dimensional evaluation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 74 references that are in the map. First posted on arXiv 1 Aug 2026.
Builds on
26 of 74 references.
References · 26
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
14 more
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
- 2020A Primer in BERTology: What we know about how BERT workscited by 807
- 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
- 2019Green AIcited by 620
- 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
- 2022Data Contamination: From Memorization to Exploitationcited by 234
- 2024AI Agents That Mattercited by 199
- 2023Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawscited by 155
- 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
- 2021Why AI is Harder Than We Thinkcited by 40
- 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturationcited by 29
- 2025Beyond Accuracy: A Multi-Dimensional Framework for Evaluating Enterprise Agentic AI Systemscited by 6
Built on by
Not cited yet.