mlsanityData to 14 Oct 2026

Auditing Future-Memory Use in Point-in-Time Language Agents

arXiv 2610.18602 ↗First posted 13 Oct 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Point-in-time (PIT) agents can recall post-cutoff facts even when their external evidence respects a historical cutoff. We examine whether this access predicts evidence gathering in 48 historical company-comparison tasks. Independent closed-book revenue probes identify the better-remembered company for each model on tasks with unequal recall, fixing the predicted direction before agent runs. We then compare paired runs with real or anonymized company identities while keeping external evidence PIT-valid. Both Qwen and DeepSeek recalled future facts, but neither supported the prespecified shift toward the better-remembered company. Qwen's four allocation estimates on 28 unequal-recall tasks were negative, with descriptive intervals spanning zero. A prospective DeepSeek replication showed an inverse pattern across all four metrics on 20 such tasks, with intervals below zero. These estimates remained negative after each company, pair, and future-fact omission, and all four post-hoc max-T-adjusted sign-flip p-values were below .05. In a prospectively specified follow-up, explicit future facts produced small estimated shifts away from the supplied-fact company, with intervals spanning zero, while final choices shifted toward it. The mechanism behind the natural inverse pattern remains unresolved. These findings support separate assessments of memory accessibility, search allocation, and final outputs alongside cutoff checks on external evidence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

19 of 26 references.

References · 19

  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  2. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  3. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  4. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  5. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
  6. 2025Profit Mirage: Revisiting Information Leakage in LLM-based Financial Agentscited by 14
  7. 2026Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Financecited by 13
  8. 2026From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Marketscited by 11
  9. 2025Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffscited by 10
  10. 2025ExAnte: A Benchmark for Ex-Ante Inference in Large Language Modelscited by 6
  11. 2026The Impact of Post-training on Data Contaminationcited by 5
  12. 2026Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMscited by 4
7 more
  1. 2026Is Deep Research Reliable? Misleading Knowledge Induces False Conclusionscited by 4
  2. 2026Knowing When to Ask: Segment-Level Credit Assignment for LLM Tool Usecited by 3
  3. 2026All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtestingcited by 2
  4. 2026Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancementcited by 2
  5. 2026FinanceHarness: Autonomous Financial Deep Research Frameworkcited by 2
  6. 2026Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scorescited by 2
  7. 2026TEMPO: Temporal Enforcement via Mode-Separated Policy Optimization for Trustworthy LLM Backtestingcited by 1

Built on by

Not cited yet.