# Forecasting With LLMs: Improved Generalization Through Feature Steering

arXiv: https://arxiv.org/abs/2606.27199 · HTML: https://mlsanity.com/paper/2606.27199/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Builds on (most cited first, 10 of 10)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Steering Language Model Refusal with Sparse Autoencoders](https://mlsanity.com/paper/2411.11296/) (2024)
- [Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis](https://mlsanity.com/paper/2309.17322/) (2023)
- [Time Awareness in Large Language Models: Benchmarking Fact Recall Across Time](https://mlsanity.com/paper/2409.13338/) (2024)
- [Evaluating LLMs in Finance Requires Explicit Bias Consideration](https://mlsanity.com/paper/2602.14233/) (2026)
- [A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMs](https://mlsanity.com/paper/2512.06607/) (2025)
- [Time Machine GPT](https://mlsanity.com/paper/2404.18543/) (2024)
- [Divergence Decoding: Inference-Time Unlearning via Auxiliary Models](https://mlsanity.com/paper/2605.31293/) (2026)

## Abstract (arXiv)

Successful forecasting involves identifying patterns between historical and future states of the world which generalize to future observations. We apply LLMs to a variety of forecasting tasks and inspect their internal states using sparse autoencoders to understand whether they appear to rely on time-specific pieces of knowledge versus generalizable patterns. Our analyses identify features associated with both time-aware reasoning and look-ahead-biased reasoning. We then apply the LLMs to an entirely different domain and intervene on these features. We find that amplifying time-awareness features substantially reduces look-ahead bias on forecasting prompts while preserving general reasoning performance. In contrast, steering the candidate look-ahead-bias features does not produce an effect. These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded reasoning.