NoLBERT: A No Lookahead(back) Foundational Language Model
Not on the map. Why?
Abstract (arXiv)
We present NoLBERT, a lightweight, timestamped foundational language model for empirical research -- particularly for forecasting in economics, finance, and the social sciences. By pretraining exclusively on text from 1976 to 1995, NoLBERT avoids both lookback and lookahead biases (information leakage) that can undermine econometric inference. It exceeds domain-specific baselines on NLP benchmarks while maintaining temporal consistency. Applied to patent texts, NoLBERT enables the construction of firm-level innovation networks and shows that gains in innovation centrality predict higher long-run profit growth.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 2 of its 4 references that are in the map. First posted on arXiv 1 Sep 2025.
Builds on
2 of 4 references.
References · 2
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
- 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 385
Built on by
Not cited yet.