# LENS: Large Pre-trained Transformer for Exploring Financial Time Series Regularities

arXiv: https://arxiv.org/abs/2408.10111 · HTML: https://mlsanity.com/paper/2408.10111/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [FinGPT: Open-Source Financial Large Language Models](https://mlsanity.com/paper/2306.06031/) (2023)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [Deep Learning for Financial Applications : A Survey](https://mlsanity.com/paper/2002.05786/) (2020)
- [Towards Representation Alignment and Uniformity in Collaborative Filtering](https://mlsanity.com/paper/2206.12811/) (2022)

## Built on by (most cited first, 3 of 3)

- [Kronos: A Foundation Model for the Language of Financial Markets](https://mlsanity.com/paper/2508.02739/) (2025)
- [HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training](https://mlsanity.com/paper/2512.20272/) (2025)
- [KiT: A Foundation Model for Financial Time-Series Forecasting using DiffusionTransformers](https://mlsanity.com/paper/2609.34507/) (2026)

## Abstract (arXiv)

Modeling large-scale time series has gained significant attention in recent years. However, its direct application in finance remains challenging due to substantial differences in data characteristics across domains. Specifically, financial systems feature inherent stochasticity and low signal-to-noise ratios, rendering traditional methods and pre-training approaches ineffective. This underscores the urgent need for a foundation model tailored to financial time series. To bridge this gap, we propose LENS, a pre-trained model for this domain. LENS effectively captures the complexity of financial stochastic systems through a carefully crafted model architecture and mitigates noise during pre-training by using an invertible embedding module. We provide a rigorous theoretical explanation of the model's effectiveness and validate its performance through extensive experiments. Pre-trained on a dataset comprising 100 billion financial observations, LENS achieves exceptional results across a wide range of critical downstream tasks. Moreover, our work offers practical insights into developing pre-trained time series models in high-noise environments, paving the way for further advancements in this pivotal research domain.