# A Human-Centric Evaluation of a Retrieval-Augmented Generation System for Explaining Quebec Insurance Contracts

arXiv: https://arxiv.org/abs/2607.15963 · HTML: https://mlsanity.com/paper/2607.15963/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Legal and financial RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Placed in

- Retrieval-augmented generation / [Legal and financial RAG](https://mlsanity.com/rag/legal-and-financial-rag/)

## Builds on (most cited first, 6 of 6)

- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain](https://mlsanity.com/paper/2408.10343/) (2024)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [Generating Intelligible Plumitifs Descriptions: Use Case Application with Ethical Considerations](https://mlsanity.com/paper/2011.12183/) (2020)
- [A User-Centred Framework for Explainable Artificial Intelligence in Human-Robot Interaction](https://mlsanity.com/paper/2109.12912/) (2021)
- [Quebec Automobile Insurance Question-Answering With Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.09623/) (2024)

## Abstract (arXiv)

With the rise of online insurance sales, consumers face a significant enquoteadvice gap, requiring them to navigate complex legal contracts without expert guidance. This paper presents a human-centric, extrinsic evaluation of a state-of-the-art Retrieval-Augmented Generation system, designed to make Quebec automobile insurance contracts more understandable. Through a user study with 154 participants from Laval University, we assess the agent's real-world utility by measuring system satisfaction, cognitive effort, perceived autonomy, and risk. Our results show the system is perceived as a enquotecognitive equalizer, receiving high ratings for satisfaction, trust, and clarity. Crucially, users value the sense of autonomy the system provides even more than the knowledge itself, with this effect being most pronounced among participants with lower financial literacy, demonstrating how such an agent can directly empower individuals. However, our study also highlights some limitations: participants expressed a strong preference for human agents in high-stakes or emotionally charged scenarios. This work highlights the importance of human-in-the-loop frameworks in ensuring the responsible implementation of AI in high-stakes consumer finance.