# INSEva: A Comprehensive Chinese Benchmark for Large Language Models in Insurance

arXiv: https://arxiv.org/abs/2509.04455 · HTML: https://mlsanity.com/paper/2509.04455/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Applying Deep Learning to Answer Selection: A Study and An Open Task](https://mlsanity.com/paper/1508.01585/) (2015)
- [Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance](https://mlsanity.com/paper/2502.18772/) (2025)
- [INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance](https://mlsanity.com/paper/2406.09105/) (2024)

## Built on by (most cited first, 3 of 3)

- [INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models](https://mlsanity.com/paper/2607.24273/) (2026)
- [Foundations and Architectures of Artificial Intelligence for Motor Insurance](https://mlsanity.com/paper/2603.18508/) (2026)
- [InsClaimBench: Benchmarking Insurance Claim Adjudication Across the Decision Chain](https://mlsanity.com/paper/2610.09671/) (2026)

## Abstract (arXiv)

Insurance, as a critical component of the global financial system, demands high standards of accuracy and reliability in AI applications. While existing benchmarks evaluate AI capabilities across various domains, they often fail to capture the unique characteristics and requirements of the insurance domain. To address this gap, we present INSEva, a comprehensive Chinese benchmark specifically designed for evaluating AI systems' knowledge and capabilities in insurance. INSEva features a multi-dimensional evaluation taxonomy covering business areas, task formats, difficulty levels, and cognitive-knowledge dimension, comprising 38,704 high-quality evaluation examples sourced from authoritative materials. Our benchmark implements tailored evaluation methods for assessing both faithfulness and completeness in open-ended responses. Through extensive evaluation of 8 state-of-the-art Large Language Models (LLMs), we identify significant performance variations across different dimensions. While general LLMs demonstrate basic insurance domain competency with average scores above 80, substantial gaps remain in handling complex, real-world insurance scenarios. The benchmark will be public soon.