mlsanityData to 7 Oct 2026

XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters

arXiv 2305.12002 ↗First posted 19 May 2023cs.CLcited by 92

Not on the map. Why?

Abstract (arXiv)

In recent years, pre-trained language models have undergone rapid development with the emergence of large-scale models. However, there is a lack of open-sourced chat models specifically designed for the Chinese language, especially in the field of Chinese finance, at the scale of hundreds of billions. To address this gap, we introduce XuanYuan 2.0, the largest Chinese chat model to date, built upon the BLOOM-176B architecture. Additionally, we propose a novel training method called hybrid-tuning to mitigate catastrophic forgetting. By combining general-domain with domain-specific knowledge and integrating the stages of pre-training and fine-tuning, XuanYuan 2.0 is capable of providing accurate and contextually appropriate responses in the Chinese financial domain.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

17 of 31 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  10. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  11. 2023BloombergGPT: A Large Language Model for Financecited by 923
  12. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
5 more
  1. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  2. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  3. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  4. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  5. 2021Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinesecited by 45

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 39; 2025: 29; 2026: 10.

92 papers.

Sort:

By area:

  1. 2023A Comprehensive Overview of Large Language Modelscited by 639
  2. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 298
  3. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  4. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  5. 2023Large Language Models in Finance: A Surveycited by 223
  6. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  7. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 182
  8. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  9. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 133
  10. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  11. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82
  12. 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
38 more
  1. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 59
  2. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  3. 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Modelscited by 52
  4. 2023Data-Juicer: A One-Stop Data Processing System for Large Language Modelscited by 52
  5. 2023EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commercecited by 48
  6. 2025Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Surveycited by 47
  7. 2023TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Modelscited by 45
  8. 2025Baichuan-M1: Pushing the Medical Capability of Large Language Modelscited by 41
  9. 2024TelecomGPT: A Framework to Build Telecom-Specfic Large Language Modelscited by 38
  10. 2024Large Language Models in Cybersecurity: State-of-the-Artcited by 34
  11. 2024A Survey of Useful LLM Evaluationcited by 28
  12. 2024MarS: a Financial Market Simulation Engine Powered by Generative Foundation Modelcited by 27
  13. 2024UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMscited by 24
  14. 2023Align on the Fly: Adapting Chatbot Behavior to Established Normscited by 22
  15. 2023PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensationcited by 17
  16. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  17. 2025Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrationscited by 16
  18. 2025Large Language Models for Computer-Aided Design: A Surveycited by 16
  19. 2024ChIRAAG: ChatGPT Informed Rapid and Automated Assertion Generationcited by 15
  20. 2025MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Applicationcited by 14
  21. 2025Trading-R1: Financial Trading with LLM Reasoning via Reinforcement Learningcited by 14
  22. 2024Re-TASK: Revisiting LLM Tasks from Capability, Skill, and Knowledge Perspectivescited by 13
  23. 2024Unearthing Large Scale Domain-Specific Knowledge from Public Corporacited by 12
  24. 2024CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Modelscited by 12
  25. 2023CFGPT: Chinese Financial Assistant with Large Language Modelcited by 11
  26. 2024FinDABench: Benchmarking Financial Data Analysis Ability of Large Language Modelscited by 11
  27. 2023MatChat: A Large Language Model and Application Service Platform for Materials Sciencecited by 10
  28. 2024Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Modelscited by 10
  29. 2024The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Modelscited by 10
  30. 2023AviationGPT: A Large Language Model for the Aviation Domaincited by 9
  31. 2024A Survey on Symbolic Knowledge Distillation of Large Language Modelscited by 9
  32. 2024IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMscited by 9
  33. 2025From Deep Learning to LLMs: A survey of AI in Quantitative Investmentcited by 9
  34. 2026RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?cited by 9
  35. 2024PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrievalcited by 8
  36. 2025FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domaincited by 8
  37. 2024No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarkscited by 7
  38. 2025Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selectioncited by 7

50 most cited of 92 All 92 (JSON)