mlsanityData to 7 Oct 2026

Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models

arXiv 2309.17050 ↗First posted 29 Sep 2023cs.CLcited by 35

On the map: Retrieval-augmented generation → Legal and financial RAG

Abstract (arXiv)

Many individuals are likely to face a legal dispute at some point in their lives, but their lack of understanding of how to navigate these complex issues often renders them vulnerable. The advancement of natural language processing opens new avenues for bridging this legal literacy gap through the development of automated legal aid systems. However, existing legal question answering (LQA) approaches often suffer from a narrow scope, being either confined to specific legal domains or limited to brief, uninformative responses. In this work, we propose an end-to-end methodology designed to generate long-form answers to any statutory law questions, utilizing a "retrieve-then-read" pipeline. To support this approach, we introduce and release the Long-form Legal Question Answering (LLeQA) dataset, comprising 1,868 expert-annotated legal questions in the French language, complete with detailed answers rooted in pertinent legal provisions. Our experimental results demonstrate promising performance on automatic evaluation metrics, but a qualitative analysis uncovers areas for refinement. As one of the only comprehensive, expert-annotated long-form LQA dataset, LLeQA has the potential to not only accelerate research towards resolving a significant real-world issue, but also act as a rigorous benchmark for evaluating NLP models in specialized domains. We publicly release our code, data, and models.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Legal and financial RAG, one of 15 directions in Retrieval-augmented generation. Builds on 60 of its 91 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

60 of 91 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
38 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  5. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  11. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  12. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  13. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  14. 2019CamemBERT: a Tasty French Language Modelcited by 368
  15. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  16. 2016Rationalizing Neural Predictionscited by 281
  17. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  18. 2019Neural Legal Judgment Prediction in Englishcited by 208
  19. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 192
  20. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  21. 2020Measuring Association Between Labels and Free-Text Rationalescited by 159
  22. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  23. 2019Interpretable Neural Predictions with Differentiable Binary Variablescited by 126
  24. 2019JEC-QA: A Legal-Domain Question Answering Datasetcited by 115
  25. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  26. 2018The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standardscited by 97
  27. 2020NILE : Natural Language Inference with Faithful Natural Language Explanationscited by 96
  28. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  29. 2017On Sampling Strategies for Neural Network-based Collaborative Filteringcited by 71
  30. 2019Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanationscited by 71
  31. 2020An Information Bottleneck Approach for Controlling Conciseness in Rationale Extractioncited by 69
  32. 2018Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMscited by 67
  33. 2019Question Answering for Privacy Policies: Combining Computational and Legal Perspectivescited by 65
  34. 2017Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generationcited by 61
  35. 2020A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answeringcited by 61
  36. 2022Legal Prompt Engineering for Multilingual Legal Judgement Predictioncited by 54
  37. 2017Improved Neural Relation Detection for Knowledge Base Question Answeringcited by 44
  38. 2017Diversity driven Attention Model for Query-based Abstractive Summarizationcited by 42

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 17; 2026: 12.

35 papers.

Sort:

By area:

  1. 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 61
  2. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  3. 2025A Reasoning-Focused Legal Retrieval Benchmarkcited by 37
  4. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimizationcited by 29
  5. 2025JuDGE: Benchmarking Judgment Document Generation for Chinese Legal Systemcited by 18
  6. 2025Benchmarking Multi-Step Legal Reasoning and Analyzing Chain-of-Thought Effects in Large Language Modelscited by 9
  7. 2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
  8. 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
  9. 2025REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processingcited by 4
  10. 2024CSS: Contrastive Semantic Similarity for Uncertainty Quantification of LLMscited by 3
  11. 2024Exploring Possibilities of AI-Powered Legal Assistance in Bangladesh through Large Language Modelingcited by 3
  12. 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3
23 more
  1. 2023Justifiable Artificial Intelligence: Engineering Large Language Models for Legal Applicationscited by 2
  2. 2025Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codescited by 2
  3. 2025Modeling the Diachronic Evolution of Legal Norms: An LRMoo-Based, Component-Level, Event-Centric Approach to Legal Knowledge Graphscited by 2
  4. 2026LegalCiteBench: Evaluating Citation Reliability in Legal Language Modelscited by 2
  5. 2026Fine-grained Claim-level RAG Benchmark for Lawcited by 2
  6. 2025EvidenceMap: Learning Evidence Analysis to Unleash the Power of Small Language Models for Biomedical Question Answeringcited by 1
  7. 2025From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertisecited by 1
  8. 2025Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknessescited by 1
  9. 2025Build AI Assistants using Large Language Models and Agents to Enhance the Engineering Education of Biomechanicscited by 1
  10. 2026Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directionscited by 1
  11. 2026NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Modelscited by 1
  12. 2024eagerlearners at SemEval2024 Task 5: The Legal Argument Reasoning Task in Civil Procedurecited by 0
  13. 2025Conversational Text Extraction with Large Language Models Using Retrieval-Augmented Systemscited by 0
  14. 2025Vision Meets Language: A RAG-Augmented YOLOv8 Framework for Coffee Disease Diagnosis and Farmer Assistancecited by 0
  15. 2025Retrieval-Augmented Generation Systems for Intellectual Property via Synthetic Multi-Angle Fine-tuningcited by 0
  16. 2025Reservoir Computing inspired Matrix Multiplication-free Language Modelcited by 0
  17. 2026Benchmarking Large Language Models for Quebec Insurance: From Closed-Book to Retrieval-Augmented Generationcited by 0
  18. 2026M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generationcited by 0
  19. 2026A Human-Centric Evaluation of a Retrieval-Augmented Generation System for Explaining Quebec Insurance Contractscited by 0
  20. 2026Rhetorical-Role-Aware Retrieval-Augmented Generation for Legal Question Answering over Indian Supreme Court Judgmentscited by 0
  21. 2026How Much Do Legal RAG Systems Still Hallucinate?cited by 0
  22. 2026Beyond Accuracy and Surface Fluency: Risk-Sensitive Evaluation of LLMs for Legal Clause Generationcited by 0
  23. 2026Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Decompose-and-Refine: Structured Legal Question Answering with Parametric Retrieval
  2. 2025NitiBench: A Comprehensive Study of LLM Framework Capabilities for Thai Legal Question Answering
  3. 2026AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System
  4. 2025Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning
  5. 2024Experimenting with Legal AI Solutions: The Case of Question-Answering for Access to Justice
  6. 2024GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering
  7. 2026Retrieval Augmented Generation Framework for the Nepali Legal Domain Question Answering
  8. 2025A Llama walks into the 'Bar': Efficient Supervised Fine-Tuning for Legal Reasoning in the Multi-state Bar Exam