mlsanityData to 7 Oct 2026

What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

arXiv 2607.08046 ↗First posted 9 Jul 2026cs.CL cs.AIcited by 2

On the map: LLM reasoning → Time series reasoning

Abstract (arXiv)

Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-Air. We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the prompt often changes the model's forecast while leaving the reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation's influence. Finally, forced answering reveals that forecasts are largely fixed before reasoning begins: a single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Time series reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 23 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

20 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Scaling Open-Ended Reasoning to Predict the Future31 Dec

Other references · 19

  1. 2024A Survey on LLM-as-a-Judgecited by 1,384
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  3. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  4. 2024Chronos: Learning the Language of Time Seriescited by 712
  5. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  6. 2023A decoder-only foundation model for time-series forecastingcited by 650
  7. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  8. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
  9. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  10. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  11. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  12. 2024Approaching Human-Level Forecasting with Language Modelscited by 102
7 more
  1. 2024Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracycited by 66
  2. 2022Forecasting Future World Events with Neural Networkscited by 63
  3. 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thoughtcited by 35
  4. 2024LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling lawcited by 32
  5. 2026Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewardscited by 13
  6. 2026FutureSim: Replaying World Events to Evaluate Adaptive Agentscited by 7
  7. 2026Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truthcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 0
  2. 2026The Corroboration Illusion: When More News Makes LLM Forecasts Less Truecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Calibrating Reasoning in Language Models with Internal Consistency
  2. 2026Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
  3. 2026Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
  4. 2026Forecasting With LLMs: Improved Generalization Through Feature Steering
  5. 2025Can Slow-thinking LLMs Reason Over Time? Empirical Studies in Time Series Forecasting
  6. 2023Measuring Faithfulness in Chain-of-Thought Reasoning
  7. 2024LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics
  8. 2024Reasoning and Tools for Human-Level Forecasting