mlsanityData to 7 Oct 2026

Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems

arXiv 2608.00680 ↗First posted 1 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Evaluating artificial intelligence systems has historically relied on outcome-based benchmarks that measure task accuracy, robustness, or fairness. While indispensable, these benchmarks provide limited diagnostic insight into the underlying cognitive processes that generate performance-leaving critical questions unanswered about how AI systems reason, integrate memory, manage complexity, or avoid generating false information. This paper introduces the Multi-Dimensional Assessment for AI Cognition (MAAC), a theoretically grounded framework for shifting evaluation from what text-based AI systems produce to how they think. MAAC defines nine cognitively motivated dimensions: Cognitive Load, Tool Execution, Content Quality, Memory Integration, Complexity Handling, Hallucination Control, Knowledge Transfer, Processing Efficiency, and Process-Outcome Alignment. Each dimension is grounded in established cognitive science theory-drawing on Marr's tri-level hypothesis, Baddeley's working memory model, Sweller's cognitive load theory, and unified theories of cognition. Five theoretical analyses provide initial support for the framework's coherence and empirical testability: dimension-to-theory mapping; a coverage matrix assessing breadth and non-redundancy; a formal gap analysis relative to current evaluation practice; a worked diagnostic illustration; and a set of a priori interdependency predictions for future empirical testing. MAAC provides a theoretical and operational framework for principled process-level cognitive assessment of text-based AI systems, complementing existing outcome-based benchmarks with cognitively grounded, multi-dimensional evaluation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 74 references that are in the map. First posted on arXiv 1 Aug 2026.

Builds on

26 of 74 references.

References · 26

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  12. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
14 more
  1. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  2. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  3. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  6. 2019Green AIcited by 620
  7. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  8. 2022Data Contamination: From Memorization to Exploitationcited by 234
  9. 2024AI Agents That Mattercited by 199
  10. 2023Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawscited by 155
  11. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  12. 2021Why AI is Harder Than We Thinkcited by 40
  13. 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturationcited by 29
  14. 2025Beyond Accuracy: A Multi-Dimensional Framework for Evaluating Enterprise Agentic AI Systemscited by 6

Built on by

Not cited yet.