mlsanityData to 7 Oct 2026

Cognitive Twins: Investigating Personalized Thinking Model Building and Its Performance Enhancement with Human-in-the-Loop

arXiv 2605.04761 ↗First posted 6 May 2026cs.LG cs.AI cs.HCcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This paper presents the Personalized Thinking Model (PTM), a hierarchical and interpretable learner representation designed for AI supported education. PTM organizes evidence from learner journals into a five-layer structure covering behavioral instances, behavioral patterns, cognitive routines, metacognitive tendencies, and self-system values. PTM is grounded in Marzano's New Taxonomy of Educational Objectives and tries to clone learner's thinking model and build cognitive twin. It was constructed using a pipeline that combines large language model inference (Gemini 2.5 Pro), sentence embeddings, dimensionality reduction, and consensus clustering. This paper evaluates PTM fidelity through three methods applied to 40 participants in a seven-week study. First, automatic evaluation using atomic information point matching yielded an overall F1 score of 74.57% before human-in-the-loop (HITL) refinement and 75.48% after refinement. Second, user evaluation using a Likert scale produced mean ratings of 4.26 and 4.30 on a five-point scale for pre and post-HITL conditions respectively. Third, semantic alignment verification showed that topic coherence increased from 0.436 at the behavioral layer to 0.626 at the core value layer, while lexical overlap with journal vocabulary decreased from 0.114 to 0.007 across those same layers. These results suggest that the PTM produces outputs with acceptable fidelity, was generally perceived by users as reflecting their thinking, and showed a pattern consistent with semantic abstraction across layers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 3 of its 49 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

3 of 49 references.

References · 3

  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2021Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherencecited by 74
  3. 2023Zero-shot Unsupervised Transfer Instance Segmentationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident
  2. 2026Can Large Language Models Simulate Human Cognition Beyond Behavioral Imitation?
  3. 2025Cognitive Foundations for Reasoning and Their Manifestation in LLMs
  4. 2024The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
  5. 2025Uncovering the Computational Ingredients of Human-Like Representations in LLMs
  6. 2026How effective are VLMs in assisting humans in inferring the quality of mental models from Multimodal short answers?
  7. 2026Hypothesis-guided discovery of cognitive algorithms via program refinement
  8. 2025Simulating Human-Like Learning Dynamics with LLM-Empowered Agents