mlsanityData to 14 Oct 2026

Forecasting With LLMs: Improved Generalization Through Feature Steering

arXiv 2606.27199 ↗First posted 25 Jun 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Successful forecasting involves identifying patterns between historical and future states of the world which generalize to future observations. We apply LLMs to a variety of forecasting tasks and inspect their internal states using sparse autoencoders to understand whether they appear to rely on time-specific pieces of knowledge versus generalizable patterns. Our analyses identify features associated with both time-aware reasoning and look-ahead-biased reasoning. We then apply the LLMs to an entirely different domain and intervene on these features. We find that amplifying time-awareness features substantially reduces look-ahead bias on forecasting prompts while preserving general reasoning performance. In contrast, steering the candidate look-ahead-bias features does not produce an effect. These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 21 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

10 of 21 references.

References · 10

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  2. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  3. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  4. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  5. 2023Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysiscited by 31
  6. 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Timecited by 13
  7. 2026Evaluating LLMs in Finance Requires Explicit Bias Considerationcited by 11
  8. 2025A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMscited by 8
  9. 2024Time Machine GPTcited by 7
  10. 2026Divergence Decoding: Inference-Time Unlearning via Auxiliary Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Can Slow-thinking LLMs Reason Over Time? Empirical Studies in Time Series Forecasting
  2. 2026TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks
  3. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications
  4. 2025Time Series Forecasting via Reasoning: A Slow-Thinking Approach with Reinforcement Fine-Tuned LLMs
  5. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
  6. 2026Reasoning-Aware Training for Time Series Forecasting
  7. 2026Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis
  8. 2024Implicit Reasoning in Deep Time Series Forecasting