Aletheia: Quantifying Cognitive Conviction in Reasoning Models via Regularized Inverse Confusion Matrix
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
In the progressive journey toward Artificial General Intelligence (AGI), current evaluation paradigms face an epistemological crisis. Static benchmarks measure knowledge breadth but fail to quantify the depth of belief. While Simhi et al. (2025) defined the CHOKE phenomenon in standard QA, we extend this framework to quantify "Cognitive Conviction" in System 2 reasoning models. We propose Project Aletheia, a cognitive physics framework that employs Tikhonov Regularization to invert the judge's confusion matrix. To validate this methodology without relying on opaque private data, we implement a Synthetic Proxy Protocol. Our preliminary pilot study on 2025 baselines (e.g., DeepSeek-R1, OpenAI o1) suggests that while reasoning models act as a "cognitive buffer," they may exhibit "Defensive OverThinking" under adversarial pressure. Furthermore, we introduce the Aligned Conviction Score (S_aligned) to verify that conviction does not compromise safety. This work serves as a blueprint for measuring AI scientific integrity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 13 references that are in the map. First posted on arXiv 4 Jan 2026.
Builds on
6 of 13 references.
References · 6
- 2025SycEval: Evaluating LLM Sycophancycited by 108
- 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 96
- 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
- 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 27
- 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 27
- 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Inverse Scaling in Test-Time Compute
- 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
- 2026Reasoning or Rationalization? The Role of Justifications in Masked Diffusion Models for Fact Verification
- 2025Catch Me If You Can: How Smaller Reasoning Models Pretend to Reason with Mathematical Fidelity
- 2026ProjectionBench: Evaluating Scientific Hypothesis Generation in LLMs Under Progressive Information Disclosure
- 2026Pramana: Fine-Tuning Large Language Models for Epistemic Reasoning through Navya-Nyaya
- 2026When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning
- 2025Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations