# Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis

arXiv: https://arxiv.org/abs/2305.07972 · HTML: https://mlsanity.com/paper/2305.07972/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 47 references that are in the map.
- First posted on arXiv 13 May 2023.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Benchmarking Machine Learning Models to Predict Corporate Bankruptcy](https://mlsanity.com/paper/2212.12051/) (2022)

## Built on by (most cited first, 30 of 75)

- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [Task Contamination: Language Models May Not Be Few-Shot Anymore](https://mlsanity.com/paper/2312.16337/) (2023)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning](https://mlsanity.com/paper/2309.13064/) (2023)
- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models](https://mlsanity.com/paper/2310.06762/) (2023)
- [Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning](https://mlsanity.com/paper/2402.18865/) (2024)
- [FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models](https://mlsanity.com/paper/2402.10986/) (2024)
- [Demystifying Domain-adaptive Post-training for Financial LLMs](https://mlsanity.com/paper/2501.04961/) (2025)
- [FinMTEB: Finance Massive Text Embedding Benchmark](https://mlsanity.com/paper/2502.10990/) (2025)
- [Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks](https://mlsanity.com/paper/2305.16633/) (2023)
- [FinEntity: Entity-level Sentiment Classification for Financial Texts](https://mlsanity.com/paper/2310.12406/) (2023)
- [Do We Need Domain-Specific Embedding Models? An Empirical Investigation](https://mlsanity.com/paper/2409.18511/) (2024)
- [Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing](https://mlsanity.com/paper/2310.12664/) (2023)
- [Forget Forgetting: Continual Learning in a World of Abundant Memory](https://mlsanity.com/paper/2502.07274/) (2025)
- [OASIS: Online Sample Selection for Continual Visual Instruction Tuning](https://mlsanity.com/paper/2506.02011/) (2025)
- [Evaluating Large Language Models as Expert Annotators](https://mlsanity.com/paper/2508.07827/) (2025)
- [PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning](https://mlsanity.com/paper/2407.21571/) (2024)
- [SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts' QA Through Six-Dimensional Feature Analysis](https://mlsanity.com/paper/2410.20651/) (2024)
- [A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classification](https://mlsanity.com/paper/2411.02476/) (2024)
- [On-Policy Replay for Continual Supervised Fine-Tuning](https://mlsanity.com/paper/2605.29495/) (2026)
- [No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks](https://mlsanity.com/paper/2403.06249/) (2024)
- [SusGen-GPT: A Data-Centric LLM for Financial NLP and Sustainability Report Generation](https://mlsanity.com/paper/2412.10906/) (2024)
- [From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery](https://mlsanity.com/paper/2605.15412/) (2026)
- [D'olares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English](https://mlsanity.com/paper/2402.07405/) (2024)
- [LLM-Measure: Generating Valid, Consistent, and Reproducible Text-Based Measures for Social Science Research](https://mlsanity.com/paper/2409.12722/) (2024)
- [CAMEF: Causal-Augmented Multi-Modality Event-Driven Financial Forecasting by Integrating Time Series Patterns and Salient Macroeconomic Announcements](https://mlsanity.com/paper/2502.04592/) (2025)
- [Numerical Claim Detection in Finance: A New Financial Dataset, Weak-Supervision Model, and Market Analysis](https://mlsanity.com/paper/2402.11728/) (2024)

## Abstract (arXiv)

Monetary policy pronouncements by Federal Open Market Committee (FOMC) are a major driver of financial market returns. We construct the largest tokenized and annotated dataset of FOMC speeches, meeting minutes, and press conference transcripts in order to understand how monetary policy influences financial markets. In this study, we develop a novel task of hawkish-dovish classification and benchmark various pre-trained language models on the proposed dataset. Using the best-performing model (RoBERTa-large), we construct a measure of monetary policy stance for the FOMC document release days. To evaluate the constructed measure, we study its impact on the treasury market, stock market, and macroeconomic indicators. Our dataset, models, and code are publicly available on Huggingface and GitHub under CC BY-NC 4.0 license.