# EDGAR-CORPUS: Billions of Tokens Make The World Go Round

arXiv: https://arxiv.org/abs/2109.14394 · HTML: https://mlsanity.com/paper/2109.14394/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 22 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 2 of 2)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)

## Built on by (most cited first, 30 of 32)

- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [FiNER: Financial Numeric Entity Recognition for XBRL Tagging](https://mlsanity.com/paper/2203.06482/) (2022)
- [Efficient Continual Pre-training for Building Domain Specific Large Language Models](https://mlsanity.com/paper/2311.08545/) (2023)
- [FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models](https://mlsanity.com/paper/2402.10986/) (2024)
- [Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training](https://mlsanity.com/paper/2506.01732/) (2025)
- [Financial Numeric Extreme Labelling: A Dataset and Benchmarking for XBRL Tagging](https://mlsanity.com/paper/2306.03723/) (2023)
- [Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance](https://mlsanity.com/paper/2308.14634/) (2023)
- [KPI-EDGAR: A Novel Dataset and Accompanying Metric for Relation Extraction from Financial Documents](https://mlsanity.com/paper/2210.09163/) (2022)
- [RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2506.00789/) (2025)
- [Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training](https://mlsanity.com/paper/2407.15892/) (2024)
- [EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial Statements](https://mlsanity.com/paper/2506.08762/) (2025)
- [Characterizing Multimodal Long-form Summarization: A Case Study on Financial Reports](https://mlsanity.com/paper/2404.06162/) (2024)
- [Financial misstatement detection: a realistic evaluation](https://mlsanity.com/paper/2305.17457/) (2023)
- [LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context](https://mlsanity.com/paper/2505.17410/) (2025)
- [Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models](https://mlsanity.com/paper/2310.13312/) (2023)
- [BeanCounter: A low-toxicity, large-scale, and open dataset of business-oriented text](https://mlsanity.com/paper/2409.17827/) (2024)
- [FinTree: Financial Dataset Pretrain Transformer Encoder for Relation Extraction](https://mlsanity.com/paper/2307.13900/) (2023)
- [BUSTER: a "BUSiness Transaction Entity Recognition" dataset](https://mlsanity.com/paper/2402.09916/) (2024)
- [OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling](https://mlsanity.com/paper/2608.05141/) (2026)
- [Next-Year Bankruptcy Prediction from Textual Data: Benchmark and Baselines](https://mlsanity.com/paper/2208.11334/) (2022)
- [Learning Semantic Text Similarity to rank Hypernyms of Financial Terms](https://mlsanity.com/paper/2303.13475/) (2023)
- [HiFi-KPI: A Dataset for Hierarchical KPI Extraction from Earnings Filings](https://mlsanity.com/paper/2502.15411/) (2025)
- [Are Companies Taking AI Risks Seriously? A Systematic Analysis of Companies' AI Risk Disclosures in SEC 10-K forms](https://mlsanity.com/paper/2508.19313/) (2025)
- [AP-OOD: Attention Pooling for Out-of-Distribution Detection](https://mlsanity.com/paper/2602.06031/) (2026)
- [Graph-based Keyword Planning for Legal Clause Generation from Topics](https://mlsanity.com/paper/2301.06901/) (2023)
- [Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Data](https://mlsanity.com/paper/2307.03410/) (2023)
- [Scaling Decentralized Learning with FLock](https://mlsanity.com/paper/2507.15349/) (2025)
- [Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift](https://mlsanity.com/paper/2605.21849/) (2026)
- [IPO-Mine: A Toolkit and Dataset for Section-Structured Analysis of Long, Multimodal IPO Documents](https://mlsanity.com/paper/2605.28714/) (2026)
- [The Stanford EDGAR Filings Dataset: Reconstructing U.S. Corporate and Financial Disclosures into Layout-Faithful and Token-Efficient Pretraining Data](https://mlsanity.com/paper/2606.18192/) (2026)

## Abstract (arXiv)

We release EDGAR-CORPUS, a novel corpus comprising annual reports from all the publicly traded companies in the US spanning a period of more than 25 years. To the best of our knowledge, EDGAR-CORPUS is the largest financial NLP corpus available to date. All the reports are downloaded, split into their corresponding items (sections), and provided in a clean, easy-to-use JSON format. We use EDGAR-CORPUS to train and release EDGAR-W2V, which are WORD2VEC embeddings for the financial domain. We employ these embeddings in a battery of financial NLP tasks and showcase their superiority over generic GloVe embeddings and other existing financial word embeddings. We also open-source EDGAR-CRAWLER, a toolkit that facilitates downloading and extracting future annual reports.