mlsanityData to 7 Oct 2026

TechGPT-2.0: A large language model project to solve the task of knowledge graph construction

arXiv 2401.04507 ↗First posted 9 Jan 2024cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Large language models have exhibited robust performance across diverse natural language processing tasks. This report introduces TechGPT-2.0, a project designed to enhance the capabilities of large language models specifically in knowledge graph construction tasks, including named entity recognition (NER) and relationship triple extraction (RTE) tasks in NLP applications. Additionally, it serves as a LLM accessible for research within the Chinese open-source model community. We offer two 7B large language model weights and a QLoRA weight specialized for processing lengthy texts.Notably, TechGPT-2.0 is trained on Huawei's Ascend server. Inheriting all functionalities from TechGPT-1.0, it exhibits robust text processing capabilities, particularly in the domains of medicine and law. Furthermore, we introduce new capabilities to the model, enabling it to process texts in various domains such as geographical areas, transportation, organizations, literary works, biology, natural sciences, astronomical objects, and architecture. These enhancements also fortified the model's adeptness in handling hallucinations, unanswerable queries, and lengthy texts. This report provides a comprehensive and detailed introduction to the full fine-tuning process on Huawei's Ascend servers, encompassing experiences in Ascend server debugging, instruction fine-tuning data processing, and model training. Our code is available at https://github.com/neukg/TechGPT-2.0

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 9 Jan 2024.

Builds on

21 of 27 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
9 more
  1. 2023Qwen Technical Reportcited by 3,273
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  3. 2020Learning to summarize from human feedbackcited by 2,562
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  5. 2014How transferable are features in deep neural networks?cited by 1,216
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  8. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  9. 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2024PersonaMark: Personalized LLM watermarking for model protection and user attributioncited by 6
  2. 2025FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphscited by 5
  3. 2025MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graphcited by 5
  4. 2025ArchSeek: Retrieving Architectural Case Studies Using Vision-Language Modelscited by 1