mlsanityData to 14 Oct 2026

Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence

arXiv 2209.02970 ↗First posted 7 Sep 2022cs.CLcited by 70

Not on the map. Why?

Abstract (arXiv)

Nowadays, foundation models become one of fundamental infrastructures in artificial intelligence, paving ways to the general intelligence. However, the reality presents two urgent challenges: existing foundation models are dominated by the English-language community; users are often given limited resources and thus cannot always use foundation models. To support the development of the Chinese-language community, we introduce an open-source project, called Fengshenbang, which leads by the research center for Cognitive Computing and Natural Language (CCNL). Our project has comprehensive capabilities, including large pre-trained models, user-friendly APIs, benchmarks, datasets, and others. We wrap all these in three sub-projects: the Fengshenbang Model, the Fengshen Framework, and the Fengshen Benchmark. An open-source roadmap, Fengshenbang, aims to re-evaluate the open-source community of Chinese pre-trained large-scale models, prompting the development of the entire Chinese large-scale model community. We also want to build a user-centered open-source ecosystem to allow individuals to access the desired models to match their computing resources. Furthermore, we invite companies, colleges, and research institutions to collaborate with us to build the large-scale open-source model-based ecosystem. We hope that this project will be the foundation of Chinese cognitive intelligence.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 39 of its 69 references that are in the map. First posted on arXiv 7 Sep 2022.

Builds on

39 of 69 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  11. 2020Longformer: The Long-Document Transformercited by 3,413
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
27 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  2. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  7. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  9. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  10. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  11. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  12. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  13. 2020On Layer Normalization in the Transformer Architecturecited by 922
  14. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  15. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  16. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  17. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  18. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  19. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  20. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  21. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58
  22. 2021FewCLUE: A Chinese Few-shot Learning Evaluation Benchmarkcited by 41
  23. 2019ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representationscited by 35
  24. 2020A Benchmark for Lease Contract Reviewcited by 26
  25. 2021ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoderscited by 22
  26. 2022Unified BERT for Few-shot Natural Language Understandingcited by 2
  27. 2022Towards No.1 in CLUE Semantic Matching Challenge: Pre-trained Language Model Erlangshen with Propensity-Corrected Losscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 33; 2024: 25; 2025: 8; 2026: 1.

70 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  2. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 481
  3. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
  4. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  5. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  6. 2023Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialoguecited by 119
  7. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 115
  8. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 85
  9. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 85
  10. 2023Watermarking Text Generated by Black-Box Language Modelscited by 76
  11. 2023GPT Can Solve Mathematical Problems Without a Calculatorcited by 68
  12. 2023Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languagescited by 52
38 more
  1. 2023SuperCLUE: A Comprehensive Chinese Large Language Model Benchmarkcited by 50
  2. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferencescited by 43
  3. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  4. 2023Origin Tracing and Detecting of LLMscited by 40
  5. 2024LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Modelscited by 37
  6. 2023Psy-LLM: Scaling up Global Mental Health Psychological Services with AI-based Large Language Modelscited by 36
  7. 2024LLMFactor: Extracting Profitable Factors through Prompts for Explainable Stock Movement Predictioncited by 33
  8. 2023AltDiffusion: A Multilingual Text-to-Image Diffusion Modelcited by 30
  9. 2024Few-shot Adaptation of Multi-modal Foundation Models: A Surveycited by 28
  10. 2023Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistantscited by 25
  11. 2024Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Modelcited by 25
  12. 2024Leveraging Large Language Model as Simulated Patients for Clinical Educationcited by 21
  13. 2024PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applicationscited by 16
  14. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  15. 2023Unveiling the Potential of Sentiment: Can Large Language Models Predict Chinese Stock Price Movements?cited by 15
  16. 2023Ziya2: Data-centric Learning is All LLMs Needcited by 15
  17. 2024BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Modelscited by 15
  18. 2022Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspectivecited by 14
  19. 2024CriticEval: Evaluating Large Language Model as Criticcited by 13
  20. 2024Qibo: A Large Language Model for Traditional Chinese Medicinecited by 13
  21. 2023Evaluating Large Language Models for Radiology Natural Language Processingcited by 12
  22. 2024BianCang: A Traditional Chinese Medicine Large Language Modelcited by 12
  23. 2024Taiyi-Diffusion-XL: Advancing Bilingual Text-to-Image Generation with Large Vision-Language Model Supportcited by 11
  24. 2024An Empirical Study and Analysis of Text-to-Image Generation Using Large Language Model-Powered Textual Representationcited by 11
  25. 2025Large Language Models Meet Legal Artificial Intelligence: A Surveycited by 10
  26. 2023Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuningcited by 9
  27. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discoverycited by 8
  28. 2024ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Modelscited by 7
  29. 2023NewsDialogues: Towards Proactive News Grounded Conversationcited by 5
  30. 2023PEA-Diffusion: Parameter-Efficient Adapter with Knowledge Distillation in non-English Text-to-Image Generationcited by 5
  31. 2024MarkPlugger: Generalizable Watermark Framework for Latent Diffusion Models without Retrainingcited by 5
  32. 2024InternLM-Law: An Open Source Chinese Legal Large Language Modelcited by 5
  33. 2023Causal Document-Grounded Dialogue Pre-trainingcited by 4
  34. 2023Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivotscited by 4
  35. 2023Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Modelscited by 4
  36. 2023iDesigner: A High-Resolution and Complex-Prompt Following Text-to-Image Diffusion Model for Interior Designcited by 4
  37. 2024From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Modelscited by 4
  38. 2024Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Modelscited by 4

50 most cited of 70 All 70 (JSON)