# Knowledge Graph Quality Evaluation under Incomplete Information

arXiv: https://arxiv.org/abs/2212.00994 · HTML: https://mlsanity.com/paper/2212.00994/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Reinforcement Knowledge Graph Reasoning for Explainable Recommendation](https://mlsanity.com/paper/1906.05237/) (2019)
- [Long Text Generation via Adversarial Training with Leaked Information](https://mlsanity.com/paper/1709.08624/) (2017)
- [Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph](https://mlsanity.com/paper/1801.10314/) (2018)
- [Adversarial Feature Matching for Text Generation](https://mlsanity.com/paper/1706.03850/) (2017)
- [GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution](https://mlsanity.com/paper/1611.04051/) (2016)
- [KBQA: Learning Question Answering over QA Corpora and Knowledge Bases](https://mlsanity.com/paper/1903.02419/) (2019)
- [Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader](https://mlsanity.com/paper/1905.07098/) (2019)
- [Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering](https://mlsanity.com/paper/1808.04126/) (2018)
- [Message Passing for Complex Question Answering over Knowledge Graphs](https://mlsanity.com/paper/1908.06917/) (2019)
- [Neural Generative Question Answering](https://mlsanity.com/paper/1512.01337/) (2015)
- [Triple Trustworthiness Measurement for Knowledge Graph](https://mlsanity.com/paper/1809.09414/) (2018)
- [Efficient Knowledge Graph Accuracy Evaluation](https://mlsanity.com/paper/1907.09657/) (2019)
- [Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases](https://mlsanity.com/paper/2010.03157/) (2020)
- [Which Knowledge Graph Is Best for Me?](https://mlsanity.com/paper/1809.11099/) (2018)

## Abstract (arXiv)

Knowledge graphs (KGs) have attracted more and more attentions because of their fundamental roles in many tasks. Quality evaluation for KGs is thus crucial and indispensable. Existing methods in this field evaluate KGs by either proposing new quality metrics from different dimensions or measuring performances at KG construction stages. However, there are two major issues with those methods. First, they highly rely on raw data in KGs, which makes KGs' internal information exposed during quality evaluation. Second, they consider more about the quality at data level instead of ability level, where the latter one is more important for downstream applications. To address these issues, we propose a knowledge graph quality evaluation framework under incomplete information (QEII). The quality evaluation task is transformed into an adversarial Q&A game between two KGs. Winner of the game is thus considered to have better qualities. During the evaluation process, no raw data is exposed, which ensures information protection. Experimental results on four pairs of KGs demonstrate that, compared with baselines, the QEII implements a reasonable quality evaluation at ability level under incomplete information.