mlsanityData to 7 Oct 2026

Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adapt

arXiv 2411.07142 ↗First posted 11 Nov 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Financial documents are filled with specialized terminology, arcane jargon, and curious acronyms that pose challenges for general-purpose text embeddings. Yet, few text embeddings specialized for finance have been reported in the literature, perhaps in part due to a lack of public datasets and benchmarks. We present BAM embeddings, a set of text embeddings finetuned on a carefully constructed dataset of 14.3M query-passage pairs. Demonstrating the benefits of domain-specific training, BAM embeddings achieve Recall@1 of 62.8% on a held-out test set, vs. only 39.2% for the best general-purpose text embedding from OpenAI. Further, BAM embeddings increase question answering accuracy by 8% on FinanceBench and show increased sensitivity to the finance-specific elements that are found in detailed, forward-looking and company and date-specific queries. To support further research we describe our approach in detail, quantify the importance of hard negative mining and dataset scale.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 11 Nov 2024.

Builds on

20 of 31 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  10. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  11. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  12. 2019Publicly Available Clinical BERT Embeddingscited by 995
8 more
  1. 2023BloombergGPT: A Large Language Model for Financecited by 923
  2. 2021Robust fine-tuning of zero-shot modelscited by 785
  3. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  4. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  5. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  6. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  7. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  8. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 4.

12 papers.

  1. 2025FinMTEB: Finance Massive Text Embedding Benchmarkcited by 18
  2. 2025Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systemscited by 9
  3. 2025PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encodercited by 5
  4. 2025T-VEC: A Telecom-Specific Vectorization Model with Enhanced Semantic Understanding via Deep Triplet Loss Fine-Tuningcited by 3
  5. 2025Adapting General-Purpose Embedding Models to Private Datasets Using Keyword-based Retrievalcited by 2
  6. 2025GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddingscited by 2
  7. 2026Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detectorcited by 2
  8. 2025PCA-RAG: Principal Component Analysis for Efficient Retrieval-Augmented Generationcited by 1
  9. 2025Adaptation of Embedding Models to Financial Filings via LLM Distillationcited by 0
  10. 2026PETRA: Transforming Web Text for Petroleum-Engineering Domain Adaptationcited by 0
  11. 2026Global Merger-Arbitrage Forecasting with Language Modelscited by 0
  12. 2026A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Modelscited by 0