mlsanityData to 7 Oct 2026

Evaluating RAG for French immigration law: a benchmark and baseline study

arXiv 2607.24449 ↗First posted 27 Jul 2026cs.IR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks. We present a publicly available benchmark and first comparative evaluation for this domain, covering permit-type recommendation, required-document retrieval, and legal citation coverage. Comparing a parametric LLM baseline against dense retrieval augmentation at two model scales (Qwen3.5-9B and -27B) on 52 annotated synthetic profiles, we find that retrieval improves administrative guidance at both scales, most notably permit-type accuracy. Our results confirm that retrieval grounding is important for more reliable administrative guidance in this domain, and motivate further investigation of hybrid retrieval strategies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 17 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

14 of 17 references.

References · 14

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  5. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  6. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328
  7. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  8. 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 113
  9. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 111
  10. 2023Explaining Legal Concepts with Augmented Large Language Models (GPT-4)cited by 39
  11. 2024LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Modelscited by 36
  12. 2024HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applicationscited by 22
2 more
  1. 2025LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversationcited by 19
  2. 2025All for law and law for all: Adaptive RAG Pipeline for Legal Researchcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Fine-grained Claim-level RAG Benchmark for Law
  2. 2026Enhancing Legal LLMs through Metadata-Enriched RAG Pipelines and Direct Preference Optimization
  3. 2025LLM-as-a-Judge: Rapid Evaluation of Legal Document Recommendation for Retrieval-Augmented Generation
  4. 2026Navigating Global AI Regulation: A Multi-Jurisdictional Retrieval-Augmented Generation System
  5. 2026Legal RAG Bench: an end-to-end benchmark for legal RAG
  6. 2025A Reasoning-Focused Legal Retrieval Benchmark
  7. 2025Towards Reliable Retrieval in RAG Systems for Large Legal Datasets
  8. 2025LRAGE: Legal Retrieval Augmented Generation Evaluation Tool