mlsanityData to 14 Oct 2026

Lawyer LLaMA Technical Report

arXiv 2305.15062 ↗First posted 24 May 2023cs.CL cs.AIcited by 54

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs), like LLaMA, have exhibited remarkable performance across various tasks. Nevertheless, when deployed to specific domains such as law or medicine, the models still confront the challenge of a deficiency in domain-specific knowledge and an inadequate capability to leverage that knowledge to resolve domain-related problems. In this paper, we propose a new framework to adapt LLMs to specific domains and build Lawyer LLaMA, a legal domain LLM, based on this framework. Specifically, we inject domain knowledge during the continual training stage and teach the model to learn professional skills using properly designed supervised fine-tuning tasks. Moreover, to alleviate the hallucination problem during the model's generation, we add a retrieval module and extract relevant legal articles before the model answers any queries. When learning domain-specific skills, we find that experts' experience is much more useful than experiences distilled from ChatGPT, where hundreds of expert-written data outperform tens of thousands of ChatGPT-generated ones. We will release our model and data.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

27 of 43 references.

References · 27

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2023BloombergGPT: A Large Language Model for Financecited by 939
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  11. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  12. 2023Instruction Tuning with GPT-4cited by 686
15 more
  1. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  2. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  3. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  4. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  5. 2019Episodic Memory in Lifelong Language Learningcited by 188
  6. 2018CAIL2018: A Large-Scale Legal Dataset for Judgment Predictioncited by 170
  7. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 170
  8. 2023DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Taskcited by 131
  9. 2019JEC-QA: A Legal-Domain Question Answering Datasetcited by 116
  10. 2021Lawformer: A Pre-trained Language Model for Chinese Legal Long Documentscited by 116
  11. 2019Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehensioncited by 81
  12. 2020Distinguish Confusing Law Articles for Legal Judgment Predictioncited by 62
  13. 2017Learning to Predict Charges for Criminal Cases with Legal Basiscited by 61
  14. 2020CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Modelcited by 36
  15. 2023Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Studycited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 22; 2025: 12; 2026: 7.

54 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  2. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 189
  3. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 178
  4. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 120
  5. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 117
  6. 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 50
  7. 2023EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commercecited by 49
  8. 2024LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generationcited by 45
  9. 2023A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detectioncited by 42
  10. 2024LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Modelcited by 34
  11. 2023Integration of Large Language Models and Federated Learningcited by 31
  12. 2024SciLitLLM: How to Adapt LLMs for Scientific Literature Understandingcited by 29
38 more
  1. 2024MarS: a Financial Market Simulation Engine Powered by Generative Foundation Modelcited by 28
  2. 2024PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluationcited by 26
  3. 2024Efficient Continual Pre-training by Mitigating the Stability Gapcited by 25
  4. 2024PLLaMa: An Open-source Large Language Model for Plant Sciencecited by 22
  5. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  6. 2023PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensationcited by 17
  7. 2024Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Financecited by 15
  8. 2024LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultationcited by 14
  9. 2024MIND: Math Informed syNthetic Dialogues for Pretraining LLMscited by 14
  10. 2025Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domainscited by 11
  11. 2025Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Designcited by 11
  12. 2024A Survey on Symbolic Knowledge Distillation of Large Language Modelscited by 9
  13. 2025LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLMcited by 8
  14. 2026HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory Systemcited by 8
  15. 2025A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiencescited by 7
  16. 2024Exploring the Nexus of Large Language Models and Legal Systems: A Short Surveycited by 6
  17. 2023FedJudge: Federated Legal Large Language Modelcited by 5
  18. 2024InternLM-Law: An Open Source Chinese Legal Large Language Modelcited by 5
  19. 2025JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoningcited by 5
  20. 2024NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysiscited by 4
  21. 2025A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directionscited by 4
  22. 2024Scalable Multi-Domain Adaptation of Language Models using Modular Expertscited by 3
  23. 2023MiChao-HuaFen 1.0: A Specialized Pre-trained Corpus Dataset for Domain-specific Large Modelscited by 2
  24. 2024Diabetica: Adapting Large Language Model to Enhance Multiple Medical Tasks in Diabetes Care and Managementcited by 2
  25. 2024LicenseGPT: A Fine-tuned Foundation Model for Publicly Available Dataset License Compliancecited by 2
  26. 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialoguescited by 2
  27. 2026Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documentscited by 2
  28. 2023HouYi: An open-source large language model specially designed for renewable energy and carbon neutrality fieldcited by 1
  29. 2023HSC-GPT: A Large Language Model for Human Settlements Constructioncited by 1
  30. 2024Quokka: An Open-source Large Language Model ChatBot for Material Sciencecited by 1
  31. 2024QCG-Rerank: Chunks Graph Rerank with Query Expansion in Retrieval-Augmented LLMs for Tourism Domaincited by 1
  32. 2025Foundations of GenIRcited by 1
  33. 2025CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysiscited by 1
  34. 2024IGOT: Information Gain Optimized Tokenizer on Domain Adaptive Pretrainingcited by 0
  35. 2025LLMPR: A Novel LLM-Driven Transfer Learning based Petition Ranking Modelcited by 0
  36. 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documentscited by 0
  37. 2025PG-CE: A Progressive Generation Dataset with Constraint Enhancement for Controllable Text Generationcited by 0
  38. 2026Chinese Labor Law Large Language Model Benchmarkcited by 0

50 most cited of 54 All 54 (JSON)