mlsanityData to 7 Oct 2026

All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting

arXiv 2602.17234 ↗First posted 19 Feb 2026cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim's decision impact, yielding Shapley-DCLR (Shapley-weighted Decision-Critical Leakage Rate) -- an interpretable metric measuring what fraction of decision-driving reasoning is contaminated. We further propose TimeSPEC (Time-Supervised Prediction with Extracted Claims), an inference-time architecture that interleaves temporally-filtered retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm retrieval and supervision are jointly necessary; and a three-task probe further illstrates that the performance cost of temporal enforcement scales with each task's reliance on post-cutoff information.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 19 Feb 2026.

Builds on

18 of 32 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  6. 2023BloombergGPT: A Large Language Model for Financecited by 923
  7. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  8. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  9. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  10. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  11. 2018The Fact Extraction and VERification (FEVER) Shared Taskcited by 108
  12. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
6 more
  1. 2022Forecasting Future World Events with Neural Networkscited by 63
  2. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
  3. 2024Is Your LLM Outdated? A Deep Look at Temporal Generalizationcited by 25
  4. 2024Provably Accurate Shapley Value Estimation via Leverage Score Samplingcited by 23
  5. 2025Profit Mirage: Revisiting Information Leakage in LLM-based Financial Agentscited by 13
  6. 2025LLMLagBench: Identifying Temporal Training Boundaries in Large Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.