mlsanityData to 7 Oct 2026

UER: An Open-Source Toolkit for Pre-training Models

arXiv 1909.05658 ↗First posted 12 Sep 2019cs.CL cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks. While there does not exist a single pre-training model that works best in all cases, it is of necessity to develop a framework that is able to deploy various pre-training models efficiently. For this purpose, we propose an assemble-on-demand pre-training toolkit, namely Universal Encoder Representations (UER). UER is loosely coupled, and encapsulated with rich modules. By assembling modules on demand, users can either reproduce a state-of-the-art pre-training model or develop a pre-training model that remains unexplored. With UER, we have built a model zoo, which contains pre-trained models based on different corpora, encoders, and targets (objectives). With proper pre-trained models, we could achieve new state-of-the-art results on a range of downstream datasets.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 19; 2023: 14; 2024: 11; 2025: 7; 2026: 4.

61 papers.

Sort:

By area:

  1. 2022A Contrastive Framework for Neural Text Generationcited by 224
  2. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
  3. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  4. 2022ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classificationcited by 66
  5. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  6. 2022CSL: A Large-scale Chinese Scientific Literature Datasetcited by 45
  7. 2021FewCLUE: A Chinese Few-shot Learning Evaluation Benchmarkcited by 41
  8. 2023DTrOCR: Decoder-only Transformer for Optical Character Recognitioncited by 40
  9. 2021LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and Generationcited by 24
  10. 2021NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Predictioncited by 23
  11. 2022TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalitiescited by 19
  12. 2022A Corpus for Understanding and Generating Moral Storiescited by 16
38 more
  1. 2025What Can String Probability Tell Us About Grammaticality?cited by 16
  2. 2022SLING: Sino Linguistic Evaluation of Large Language Modelscited by 15
  3. 2024Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialoguecited by 15
  4. 2021Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Modelcited by 14
  5. 2024Evolving Knowledge Distillation with Large Language Models and Active Learningcited by 12
  6. 2022MetaAID: A Flexible Framework for Developing Metaverse Applications via AI Technology and Human Editingcited by 11
  7. 2022Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialogcited by 11
  8. 2022Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semanticscited by 10
  9. 2023A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairmentcited by 10
  10. 2023Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modellingcited by 10
  11. 2022CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluationcited by 9
  12. 2023SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanitiescited by 9
  13. 2025Entropy-Driven Pre-Tokenization for Byte-Pair Encodingcited by 6
  14. 2022Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenariocited by 5
  15. 2023NewsDialogues: Towards Proactive News Grounded Conversationcited by 5
  16. 2023A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extractioncited by 4
  17. 2023MetaAID 2.0: An Extensible Framework for Developing Metaverse Applications via Human-controllable Pre-trained Modelscited by 3
  18. 2023Methods for Acquiring and Incorporating Knowledge into Stock Price Prediction: A Surveycited by 3
  19. 2023Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learningcited by 3
  20. 2024Inconsistent dialogue responses and how to recover from themcited by 3
  21. 2025FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classificationcited by 3
  22. 2023Prompt Pool based Class-Incremental Continual Learning for Dialog State Trackingcited by 2
  23. 2024LiFi: Lightweight Controlled Text Generation with Fine-Grained Control Codescited by 2
  24. 2025Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive zijicited by 2
  25. 2025Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Braincited by 2
  26. 2022Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matchingcited by 1
  27. 2022TSMind: Alibaba and Soochow University's Submission to the WMT22 Translation Suggestion Taskcited by 1
  28. 2023Exploring Energy-based Language Models with Different Architectures and Training Methods for Speech Recognitioncited by 1
  29. 2024LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correctioncited by 1
  30. 2024Design as Desired: Utilizing Visual Question Answering for Multimodal Pre-trainingcited by 1
  31. 2024FinTruthQA: A Benchmark for AI-Driven Financial Disclosure Quality Assessment in Investor -- Firm Interactionscited by 1
  32. 2026Hot-Start Chinese Language Modeling:Visual Glyphs Accelerate Sample-Efficient Learningcited by 1
  33. 2026Text-Preserving Lossy Text Compression: A Study of Strategic Deletion and LLM Reconstructioncited by 1
  34. 2022Semantic Matching from Different Perspectivescited by 0
  35. 2022PathSAGE: Spatial Graph Attention Neural Networks With Random Path Samplingcited by 0
  36. 2022A Benchmark for Understanding and Generating Dialogue between Characters in Storiescited by 0
  37. 2022Generation of Chinese classical poetry based on pre-trained modelcited by 0
  38. 2022A Method to Judge the Style of Classical Poetry Based on Pre-trained Modelcited by 0

50 most cited of 61 All 61 (JSON)