# RDF-Based Structured Quality Assessment Representation of Multilingual LLM Evaluations

arXiv: https://arxiv.org/abs/2504.21605 · HTML: https://mlsanity.com/paper/2504.21605/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 4 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 3 of 3)

- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?](https://mlsanity.com/paper/2401.11911/) (2024)
- [Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective](https://mlsanity.com/paper/2411.14258/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) increasingly serve as knowledge interfaces, yet systematically assessing their reliability with conflicting information remains difficult. We propose an RDF-based framework to assess multilingual LLM quality, focusing on knowledge conflicts. Our approach captures model responses across four distinct context conditions (complete, incomplete, conflicting, and no-context information) in German and English. This structured representation enables the comprehensive analysis of knowledge leakage-where models favor training data over provided context-error detection, and multilingual consistency. We demonstrate the framework through a fire safety domain experiment, revealing critical patterns in context prioritization and language-specific performance, and demonstrating that our vocabulary was sufficient to express every assessment facet encountered in the 28-question study.