mlsanityData to 7 Oct 2026

BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark

arXiv 2302.09432 ↗First posted 18 Feb 2023cs.CLcited by 60

Not on the map. Why?

Abstract (arXiv)

To advance Chinese financial natural language processing (NLP), we introduce BBT-FinT5, a new Chinese financial pre-training language model based on the T5 model. To support this effort, we have built BBT-FinCorpus, a large-scale financial corpus with approximately 300GB of raw text from four different sources. In general domain NLP, comprehensive benchmarks like GLUE and SuperGLUE have driven significant advancements in language model pre-training by enabling head-to-head comparisons among models. Drawing inspiration from these benchmarks, we propose BBT-CFLEB, a Chinese Financial Language understanding and generation Evaluation Benchmark, which includes six datasets covering both understanding and generation tasks. Our aim is to facilitate research in the development of NLP within the Chinese financial domain. Our model, corpus and benchmark are released at https://github.com/ssymmetry/BBT-FinCUGE-Applications. Our work belongs to the Big Bang Transformer (BBT), a large-scale pre-trained language model project.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 18 Feb 2023.

Builds on

18 of 34 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  8. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  9. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  10. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  11. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  12. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
6 more
  1. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  2. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 227
  3. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  4. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
  5. 2019UER: An Open-Source Toolkit for Pre-training Modelscited by 61
  6. 2021Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinesecited by 45

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 26; 2025: 20; 2026: 2.

60 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  2. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  3. 2023Large Language Models in Finance: A Surveycited by 223
  4. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  5. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92
  6. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  7. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82
  8. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  9. 2023Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answeringcited by 64
  10. 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
  11. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 59
  12. 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Modelscited by 52
38 more
  1. 2025Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Surveycited by 47
  2. 2023BizBench: A Quantitative Reasoning Benchmark for Business and Financecited by 28
  3. 2024EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoningcited by 21
  4. 2023Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Modelscited by 20
  5. 2023CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Modelcited by 18
  6. 2025FinMTEB: Finance Massive Text Embedding Benchmarkcited by 18
  7. 2024SEC-QA: A Systematic Evaluation Corpus for Financial QAcited by 17
  8. 2024Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Financecited by 15
  9. 2025Trading-R1: Financial Trading with LLM Reasoning via Reinforcement Learningcited by 14
  10. 2024CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Modelscited by 12
  11. 2024Do We Need Domain-Specific Embedding Models? An Empirical Investigationcited by 12
  12. 2023CFGPT: Chinese Financial Assistant with Large Language Modelcited by 11
  13. 2024FinDABench: Benchmarking Financial Data Analysis Ability of Large Language Modelscited by 11
  14. 2024TradExpert: Revolutionizing Trading with Mixture of Expert LLMscited by 11
  15. 2024Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Modelscited by 10
  16. 2025EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environmentscited by 8
  17. 2024No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarkscited by 7
  18. 2025SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilitiescited by 6
  19. 2024D'olares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and Englishcited by 5
  20. 2025Enterprise Large Language Model Evaluation Benchmarkcited by 5
  21. 2025FinTeam: A Multi-Agent Collaborative Intelligence System for Comprehensive Financial Scenarioscited by 5
  22. 2025Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learningcited by 5
  23. 2024Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Modelscited by 4
  24. 2025A Survey of LLM × DATAcited by 4
  25. 2025CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Modelcited by 4
  26. 2025Advancing Financial Engineering with Foundation Models: Progress, Applications, and Challengescited by 4
  27. 2023ArcGPT: A Large Language Model Tailored for Real-world Archival Applicationscited by 3
  28. 2024What Teaches Robots to Walk, Teaches Them to Trade too -- Regime Adaptive Execution using Informed Data and LLMscited by 3
  29. 2024Financial Knowledge Large Language Modelcited by 3
  30. 2025VideoConviction: A Multimodal Benchmark for Human Conviction and Stock Market Recommendationscited by 3
  31. 2026FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarioscited by 3
  32. 2024Enhancing Startup Success Predictions in Venture Capital: A GraphRAG Augmented Multivariate Time Series Methodcited by 2
  33. 2024A Dutch Financial Large Language Modelcited by 2
  34. 2025Beyond Single-Sentence Prompts: Upgrading Value Alignment Benchmarks with Dialogues and Storiescited by 2
  35. 2025LegalEval-Q: A New Benchmark for The Quality Evaluation of LLM-Generated Legal Textcited by 2
  36. 2025M3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial Meeting Understanding Evaluation Datasetcited by 2
  37. 2025FinLMM-R1: Enhancing Financial Reasoning in LMM through Scalable Data and Reward Designcited by 2
  38. 2023HouYi: An open-source large language model specially designed for renewable energy and carbon neutrality fieldcited by 1

50 most cited of 60 All 60 (JSON)