# InternLM-Law: An Open Source Chinese Legal Large Language Model

arXiv: https://arxiv.org/abs/2406.14887 · HTML: https://mlsanity.com/paper/2406.14887/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [CMMLU: Measuring massive multitask language understanding in Chinese](https://mlsanity.com/paper/2306.09212/) (2023)
- [CLUE: A Chinese Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2004.05986/) (2020)
- [OctoPack: Instruction Tuning Code Large Language Models](https://mlsanity.com/paper/2308.07124/) (2023)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services](https://mlsanity.com/paper/2309.11325/) (2023)
- [LawBench: Benchmarking Legal Knowledge of Large Language Models](https://mlsanity.com/paper/2309.16289/) (2023)
- [Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents](https://mlsanity.com/paper/2105.03887/) (2021)
- [Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension](https://mlsanity.com/paper/1904.09679/) (2019)
- [Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence](https://mlsanity.com/paper/2209.02970/) (2022)
- [ChID: A Large-scale Chinese IDiom Dataset for Cloze Test](https://mlsanity.com/paper/1906.01265/) (2019)
- [Lawyer LLaMA Technical Report](https://mlsanity.com/paper/2305.15062/) (2023)
- [A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges](https://mlsanity.com/paper/2204.04859/) (2022)
- [CSL: A Large-scale Chinese Scientific Literature Dataset](https://mlsanity.com/paper/2209.05034/) (2022)
- [LEVEN: A Large-Scale Chinese Legal Event Detection Dataset](https://mlsanity.com/paper/2203.08556/) (2022)
- [WeLM: A Well-Read Pre-trained Language Model for Chinese](https://mlsanity.com/paper/2209.10372/) (2022)
- [Learning Fine-grained Fact-Article Correspondence in Legal Cases](https://mlsanity.com/paper/2104.10726/) (2021)
- [Dependency Learning for Legal Judgment Prediction with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/2112.06370/) (2021)
- [AST-Transformer: Encoding Abstract Syntax Trees Efficiently for Code Summarization](https://mlsanity.com/paper/2112.01184/) (2021)

## Built on by (most cited first, 5 of 5)

- [An overview of domain-specific foundation model: key technologies, applications and challenges](https://mlsanity.com/paper/2409.04267/) (2024)
- [A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences](https://mlsanity.com/paper/2503.00841/) (2025)
- [Legal Evalutions and Challenges of Large Language Models](https://mlsanity.com/paper/2411.10137/) (2024)
- [ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestine](https://mlsanity.com/paper/2412.14771/) (2024)
- [CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysis](https://mlsanity.com/paper/2509.21208/) (2025)

## Abstract (arXiv)

While large language models (LLMs) have showcased impressive capabilities, they struggle with addressing legal queries due to the intricate complexities and specialized expertise required in the legal field. In this paper, we introduce InternLM-Law, a specialized LLM tailored for addressing diverse legal queries related to Chinese laws, spanning from responding to standard legal questions (e.g., legal exercises in textbooks) to analyzing complex real-world legal situations. We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries, and implement a data filtering and processing pipeline to ensure its diversity and quality. Our training approach involves a novel two-stage process: initially fine-tuning LLMs on both legal-specific and general-purpose content to equip the models with broad knowledge, followed by exclusive fine-tuning on high-quality legal data to enhance structured output generation. InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks. We make InternLM-Law and our dataset publicly available to facilitate future research in applying LLMs within the legal domain.