# Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation

arXiv: https://arxiv.org/abs/2010.12872 · HTML: https://mlsanity.com/paper/2010.12872/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales](https://mlsanity.com/paper/2211.01562/) (2022)
- [Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering](https://mlsanity.com/paper/2410.08085/) (2024)
- [Graph Structural Attack by Perturbing Spectral Distance](https://mlsanity.com/paper/2111.00684/) (2021)
- [ComFact: A Benchmark for Linking Contextual Commonsense Knowledge](https://mlsanity.com/paper/2210.12678/) (2022)
- [SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning](https://mlsanity.com/paper/2104.08793/) (2021)
- [HyperAttack: Multi-Gradient-Guided White-box Adversarial Structure Attack of Hypergraph Neural Networks](https://mlsanity.com/paper/2302.12407/) (2023)
- [On the Security Risks of Knowledge Graph Reasoning](https://mlsanity.com/paper/2305.02383/) (2023)
- [An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks](https://mlsanity.com/paper/2210.13002/) (2022)
- [Reinforcement Explanation Learning](https://mlsanity.com/paper/2111.13406/) (2021)
- [Balanced Edge Pruning for Graph Anomaly Detection with Noisy Labels](https://mlsanity.com/paper/2407.05934/) (2024)

## Abstract (arXiv)

Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over the KG, such KG-augmented models can also "explain" which KG information was most relevant for making a given prediction. In this paper, we question whether these models are really behaving as we expect. We show that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs, which maintain the downstream performance of the original KG while significantly deviating from the original KG's semantics and structure. Our findings raise doubts about KG-augmented models' ability to reason about KG information and give sensible explanations.