mlsanityData to 7 Oct 2026

PatentGPT: A Large Language Model for Intellectual Property

arXiv 2404.18255 ↗First posted 28 Apr 2024cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

In recent years, large language models(LLMs) have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) domain is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP domain. Remarkably, our model surpassed GPT-4 on the 2019 China Patent Agent Qualification Examination, scoring 65 and matching human expert levels. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 55 of its 62 references that are in the map. First posted on arXiv 28 Apr 2024.

Builds on

55 of 62 references.

References · 55

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  11. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
38 more
  1. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  5. 2024Mixtral of Expertscited by 1,785
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  13. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  14. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  15. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  16. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  17. 2023Textbooks Are All You Needcited by 561
  18. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  19. 2023Language Models Represent Space and Timecited by 306
  20. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  21. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  22. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  23. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  24. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  25. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  26. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  27. 2024Secrets of RLHF in Large Language Models Part II: Reward Modelingcited by 170
  28. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  29. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  30. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 108
  31. 2023FinGPT: Democratizing Internet-scale Data for Financial Large Language Modelscited by 65
  32. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 65
  33. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  34. 2022Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learnerscited by 51
  35. 2022In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Modelscited by 36
  36. 2023Chinese Open Instruction Generalist: A Preliminary Releasecited by 35
  37. 2024Selecting Large Language Model to Fine-tune via Rectified Scaling Lawcited by 33
  38. 2023CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Modelscited by 28

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2025Towards Better Evaluation for Generated Patent Claimscited by 8
  2. 2025Enriching Patent Claim Generation with European Patent Datasetcited by 5
  3. 2024EvoPat: A Multi-LLM-based Patents Summarization and Analysis Agentcited by 4
  4. 2025Self-Filtered Distillation with LLMs-generated Trust Indicators for Reliable Patent Classificationcited by 4
  5. 2024Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMscited by 1
  6. 2026FlowPlan-G2P: A Structured Generation Framework for Transforming Scientific Papers into Patent Descriptionscited by 1
  7. 2026Self-Knowledge Retrieval Augmented Generation Framework for Patent Matchingcited by 0
  8. 2026LogicTree-RAG: Logic Tree-guided Retrieval-Augmented Generation for Long-form Patent Draftingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025IPBench: Benchmarking the Knowledge of Large Language Models in Intellectual Property
  2. 2024IPEval: A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models
  3. 2024Large Language Model for Patent Concept Generation
  4. 2025PATENTWRITER: A Benchmarking Study for Patent Drafting with LLMs
  5. 2025LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLM
  6. 2023Large-Scale Text Analysis Using Generative Language Models: A Case Study in Discovering Public Value Expressions in AI Patents
  7. 2025Enhancing the Patent Matching Capability of Large Language Models via the Memory Graph
  8. 2023GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond