RADAR: Mechanistic Pathways for Detecting Data Contamination in LLM Evaluation
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Data contamination poses a significant challenge to reliable LLM evaluation, where models may achieve high performance by memorizing training data rather than demonstrating genuine reasoning capabilities. We introduce RADAR (Recall vs. Reasoning Detection through Activation Representation), a novel framework that leverages mechanistic interpretability to detect contamination by distinguishing recall-based from reasoning-based model responses. RADAR extracts 37 features spanning surface-level confidence trajectories and deep mechanistic properties including attention specialization, circuit dynamics, and activation flow patterns. Using an ensemble of classifiers trained on these features, RADAR achieves 93% accuracy on a diverse evaluation set, with perfect performance on clear cases and 76.7% accuracy on challenging ambiguous examples. This work demonstrates the potential of mechanistic interpretability for advancing LLM evaluation beyond traditional surface-level metrics.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 4 of its 9 references that are in the map. First posted on arXiv 10 Oct 2025.
Builds on
4 of 9 references.
References · 4
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
- 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
- 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?
- 2025RADAR: Reasoning-Ability and Difficulty-Aware Routing for Reasoning LLMs
- 2026The Illusion of Reasoning: Exposing Evasive Data Contamination in LLMs via Zero-CoT Truncation
- 2025RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration
- 2024DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoning
- 2025The Rogue Scalpel: Activation Steering Compromises LLM Safety
- 2025Test of Time: Rethinking Temporal Signal of Benchmark Contamination
- 2026MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents