mlsanityData to 7 Oct 2026

WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain

arXiv 2211.00083 ↗First posted 31 Oct 2022cs.CL cs.AI cs.LGcited by 117

Not on the map. Why?

Abstract (arXiv)

Pre-trained language models have shown impressive performance on a variety of tasks and domains. Previous research on financial language models usually employs a generic training scheme to train standard model architectures, without completely leveraging the richness of the financial data. We propose a novel domain specific Financial LANGuage model (FLANG) which uses financial keywords and phrases for better masking, together with span boundary objective and in-filing objective. Additionally, the evaluation benchmarks in the field have been limited. To this end, we contribute the Financial Language Understanding Evaluation (FLUE), an open-source comprehensive suite of benchmarks for the financial domain. These include new benchmarks across 5 NLP tasks in financial domain as well as common benchmarks used in the previous research. Experiments on these benchmarks suggest that our model outperforms those in prior literature on a variety of NLP tasks. Our models, code and benchmark data are publicly available on Github and Huggingface.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Builds on 22 of its 60 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

22 of 60 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2018Deep contextualized word representationscited by 2,285
  7. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  9. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  11. 2019Publicly Available Clinical BERT Embeddingscited by 995
  12. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
10 more
  1. 2019How to Fine-Tune BERT for Text Classification?cited by 409
  2. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  3. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  4. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  5. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  6. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  7. 2020Impact of News on the Commodity Market: Dataset and Resultscited by 80
  8. 2019Span Selection Pre-training for Question Answeringcited by 36
  9. 2020Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptationcited by 14
  10. 2018Financial Aspect-Based Sentiment Analysis using Deep Representationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 32; 2024: 40; 2025: 27; 2026: 18.

117 papers.

Sort:

By area:

  1. 2023BloombergGPT: A Large Language Model for Financecited by 923
  2. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  3. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
  4. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  5. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  6. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  7. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  8. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  9. 2023InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuningcited by 80
  10. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 77
  11. 2023Trillion Dollar Words: A New Financial Dataset, Task & Market Analysiscited by 73
  12. 2023FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscited by 68
38 more
  1. 2023TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledgecited by 62
  2. 2024INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agentcited by 62
  3. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  4. 2023Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Taskscited by 59
  5. 2023Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Modelscited by 58
  6. 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Modelscited by 52
  7. 2024Designing Heterogeneous LLM Agents for Financial Sentiment Analysiscited by 48
  8. 2025Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Surveycited by 47
  9. 2024Financial Report Chunking for Effective Retrieval Augmented Generationcited by 46
  10. 2024CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answeringcited by 45
  11. 2025StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?cited by 43
  12. 2023MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretrainingcited by 33
  13. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domaincited by 28
  14. 2024STEER: Assessing the Economic Rationality of Large Language Modelscited by 26
  15. 2024Simulating Financial Market via Large Language Model based Agentscited by 25
  16. 2024Benchmarking Large Language Models on CFLUE -- A Chinese Financial Language Understanding Evaluation Datasetcited by 23
  17. 2025FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Tradingcited by 23
  18. 2024ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networkscited by 22
  19. 2025BizFinBench: A Business-Driven Real-World Financial Benchmark for Evaluating LLMscited by 22
  20. 2024EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoningcited by 21
  21. 2023Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Modelscited by 20
  22. 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Modelscited by 18
  23. 2023REFinD: Relation Extraction Financial Datasetcited by 17
  24. 2023PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensationcited by 17
  25. 2023One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial Supportcited by 15
  26. 2023Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Taskscited by 14
  27. 2025LiveTradeBench: Seeking Real-World Alpha with Large Language Modelscited by 13
  28. 2024CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Modelscited by 12
  29. 2024Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adaptcited by 12
  30. 2023CFGPT: Chinese Financial Assistant with Large Language Modelcited by 11
  31. 2023Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processingcited by 11
  32. 2024TradExpert: Revolutionizing Trading with Mixture of Expert LLMscited by 11
  33. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  34. 2024SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts' QA Through Six-Dimensional Feature Analysiscited by 8
  35. 2024Baichuan4-Finance Technical Reportcited by 8
  36. 2025EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environmentscited by 8
  37. 2023TCM-GPT: Efficient Pre-training of Large Language Models for Domain Adaptation in Traditional Chinese Medicinecited by 7
  38. 2024No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarkscited by 7

50 most cited of 117 All 117 (JSON)