# J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain

arXiv: https://arxiv.org/abs/2503.18360 · HTML: https://mlsanity.com/paper/2503.18360/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 18 of 18)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models](https://mlsanity.com/paper/2111.02840/) (2021)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning](https://mlsanity.com/paper/2308.00436/) (2023)
- [Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue](https://mlsanity.com/paper/2308.03549/) (2023)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [LEVEN: A Large-Scale Chinese Legal Event Detection Dataset](https://mlsanity.com/paper/2203.08556/) (2022)
- [MathAttack: Attacking Large Language Models Towards Math Solving Ability](https://mlsanity.com/paper/2309.01686/) (2023)
- [EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoning](https://mlsanity.com/paper/2405.07938/) (2024)
- [STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals](https://mlsanity.com/paper/2406.15313/) (2024)
- [MUSER: A Multi-View Similar Case Retrieval Dataset](https://mlsanity.com/paper/2310.15602/) (2023)
- [Baichuan2-Sum: Instruction Finetune Baichuan2-7B Model for Dialogue Summarization](https://mlsanity.com/paper/2401.15496/) (2024)
- [LEEC: A Legal Element Extraction Dataset with an Extensive Domain-Specific Label System](https://mlsanity.com/paper/2310.01271/) (2023)

## Built on by (most cited first, 7 of 7)

- [PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice](https://mlsanity.com/paper/2601.16669/) (2026)
- [Mitigating Manipulation and Enhancing Persuasion: A Reflective Multi-Agent Approach for Legal Argument Generation](https://mlsanity.com/paper/2506.02992/) (2025)
- [EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers](https://mlsanity.com/paper/2511.06890/) (2025)
- [Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning](https://mlsanity.com/paper/2509.22472/) (2025)
- [Can LLMs Identify Tax Abuse?](https://mlsanity.com/paper/2508.20097/) (2025)
- [Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2601.15267/) (2026)
- [LEGO: Synergizing Expert GraphRAG and Expert Chain-of-Thought for Legal Reasoning](https://mlsanity.com/paper/2609.27009/) (2026)

## Abstract (arXiv)

As the scale and capabilities of Large Language Models (LLMs) increase, their applications in knowledge-intensive fields such as legal domain have garnered widespread attention. However, it remains doubtful whether these LLMs make judgments based on domain knowledge for reasoning. If LLMs base their judgments solely on specific words or patterns, rather than on the underlying logic of the language, the ''LLM-as-judges'' paradigm poses substantial risks in the real-world applications. To address this question, we propose a method of legal knowledge injection attacks for robustness testing, thereby inferring whether LLMs have learned legal knowledge and reasoning logic. In this paper, we propose J&H: an evaluation framework for detecting the robustness of LLMs under knowledge injection attacks in the legal domain. The aim of the framework is to explore whether LLMs perform deductive reasoning when accomplishing legal tasks. To further this aim, we have attacked each part of the reasoning logic underlying these tasks (major premise, minor premise, and conclusion generation). We have collected mistakes that legal experts might make in judicial decisions in the real world, such as typos, legal synonyms, inaccurate external legal statutes retrieval. However, in real legal practice, legal experts tend to overlook these mistakes and make judgments based on logic. However, when faced with these errors, LLMs are likely to be misled by typographical errors and may not utilize logic in their judgments. We conducted knowledge injection attacks on existing general and domain-specific LLMs. Current LLMs are not robust against the attacks employed in our experiments. In addition we propose and compare several methods to enhance the knowledge robustness of LLMs.