mlsanityData to 14 Oct 2026

Designing AI-Resilient Assessments Using Interconnected Problems: A Theoretically Grounded and Empirically Validated Framework

arXiv 2512.10758 ↗First posted 11 Dec 2025cs.CY cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

The proliferation of generative AI tools has rendered traditional modular assessments in computing and data-centric education increasingly ineffective, creating a disconnect between academic evaluation and authentic skill measurement. This paper presents a theoretically grounded framework for designing AI-resilient assessments, supported by formal analysis and empirical validation. We make three primary contributions. First, we establish two formal propositions. (1) Assessments composed of interconnected problems, in which outputs serve as inputs to subsequent tasks, are inherently more AI-resilient than modular assessments due to their reliance on multi-step reasoning and sustained context. (2) Semi-structured problems with deterministic success criteria provide more reliable measures of student competency than fully open-ended projects, which allow AI systems to default to familiar solution templates. These results challenge widely cited recommendations in recent institutional and policy guidance that promote open-ended assessments as inherently more robust to AI assistance. Second, we validate these propositions through empirical analysis of three university data science courses (N = 117). We observe a substantial AI inflation effect: students achieve near-perfect scores on AI-assisted modular homework, while performance drops by approximately 30 percentage points on proctored exams (Cohen d = 1.51). In contrast, interconnected projects remain strongly aligned with modular assessments (r = 0.954, p < 0.001) while maintaining AI resistance, whereas proctored exams show weaker alignment (r = 0.726, p < 0.001). Third, we translate these findings into a practical assessment design procedure that enables educators to construct evaluations that promote deeper engagement, reflect industry practice, and resist trivial AI delegation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 49 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

15 of 49 references.

References · 15

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  4. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  5. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  6. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 85
  7. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  8. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  9. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 68
  10. 2024Demystifying Verbatim Memorization in Large Language Modelscited by 56
  11. 2023Large Language Models Are Not Strong Abstract Reasonerscited by 53
  12. 2024Creativity Has Left the Chat: The Price of Debiasing Language Modelscited by 29
3 more
  1. 2024Promises and challenges of generative artificial intelligence for human learningcited by 17
  2. 2024Generative AI in Higher Education: A Global Perspective of Institutional Adoption Policies and Guidelinescited by 13
  3. 2025Policy Optimization Prefers The Path of Least Resistancecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.