# PALM: Point-in-Time Adaptation for Financial Language Models

arXiv: https://arxiv.org/abs/2609.30316 · HTML: https://mlsanity.com/paper/2609.30316/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)
- [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022)
- [Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models](https://mlsanity.com/paper/2304.07619/) (2023)
- [Time Waits for No One! Analysis and Challenges of Temporal Misalignment](https://mlsanity.com/paper/2111.07408/) (2021)
- [Is Your LLM Outdated? A Deep Look at Temporal Generalization](https://mlsanity.com/paper/2405.08460/) (2024)
- [Reasoning or Overthinking: Evaluating Large Language Models on Financial Sentiment Analysis](https://mlsanity.com/paper/2506.04574/) (2025)
- [FinDPO: Financial Sentiment Analysis for Algorithmic Trading through Preference Optimization of LLMs](https://mlsanity.com/paper/2507.18417/) (2025)
- [FinSTaR: Towards Financial Reasoning with Time Series Reasoning Models](https://mlsanity.com/paper/2605.03460/) (2026)
- [Scaling Point-in-Time Language Models: Enabling Economic Evaluation of Embeddings](https://mlsanity.com/paper/2607.11889/) (2026)
- [Extracting the Structure of Press Releases for Predicting Earnings Announcement Returns](https://mlsanity.com/paper/2509.24254/) (2025)

## Abstract (arXiv)

Language models used in financial backtests suffer from look-ahead bias, as a model trained on text published after the study period has already observed the outcomes it is asked to predict. To handle this issue, point-in-time (PIT) language models are pretrained on chronologically filtered corpora and released as one checkpoint per calendar year, each with a documented cutoff. However, each additional year costs a full pretraining run, and whether that run is necessary has never been tested. In this paper, we show that the annual pretraining run is not necessary. We instead compare each checkpoint against the newer one that replaced it, and find that the newer checkpoint scores no better on the same evaluation window. Motivated by this observation, we propose PALM (Point-in-time Adaptation for financial Language Models), a simple yet effective alternative to annual pretraining that fits a low-rank adapter on text published before the decision date without modifying any pretrained weight. We further find that a small adapter is enough to add a new period to the knowledge an old checkpoint already encodes, and that this outperforms continued pretraining. We validate PALM on a decade of financial news and on various families of PIT models, whose cutoffs span two decades and whose sizes range from 1.3 to 4.2B. Code is available at: https://github.com/seunghan96/palm.