# A Copy Mechanism for Handling Knowledge Base Elements in SPARQL Neural Machine Translation

arXiv: https://arxiv.org/abs/2211.10271 · HTML: https://mlsanity.com/paper/2211.10271/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 18 Nov 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data](https://mlsanity.com/paper/1903.00138/) (2019)
- [Modern Baselines for SPARQL Semantic Parsing](https://mlsanity.com/paper/2204.12793/) (2022)
- [Neural Machine Translating from Natural Language to SPARQL](https://mlsanity.com/paper/1906.09302/) (2019)
- [DuoRAT: Towards Simpler Text-to-SQL Models](https://mlsanity.com/paper/2010.11119/) (2020)
- [Content Enhanced BERT-based Text-to-SQL Generation](https://mlsanity.com/paper/1910.07179/) (2019)
- [Outlining and Filling: Hierarchical Query Graph Generation for Answering Complex Questions over Knowledge Graphs](https://mlsanity.com/paper/2111.00732/) (2021)
- [SPARQL as a Foreign Language](https://mlsanity.com/paper/1708.07624/) (2017)
- [Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?](https://mlsanity.com/paper/2205.06573/) (2022)
- [Knowledge Graph Question Answering using Graph-Pattern Isomorphism](https://mlsanity.com/paper/2103.06752/) (2021)

## Built on by (most cited first, 4 of 4)

- [Assessing SPARQL capabilities of Large Language Models](https://mlsanity.com/paper/2409.05925/) (2024)
- [A Comprehensive Evaluation of Neural SPARQL Query Generation from Natural Language Questions](https://mlsanity.com/paper/2304.07772/) (2023)
- [SPARKLE: Enhancing SPARQL Generation with Direct KG Integration in Decoding](https://mlsanity.com/paper/2407.01626/) (2024)
- [Text-to-SPARQL Generation with Reinforcement Learning: A GRPO-based Approach on DBLP](https://mlsanity.com/paper/2605.20066/) (2026)

## Abstract (arXiv)

Neural Machine Translation (NMT) models from English to SPARQL are a promising development for SPARQL query generation. However, current architectures are unable to integrate the knowledge base (KB) schema and handle questions on knowledge resources, classes, and properties unseen during training, rendering them unusable outside the scope of topics covered in the training set. Inspired by the performance gains in natural language processing tasks, we propose to integrate a copy mechanism for neural SPARQL query generation as a way to tackle this issue. We illustrate our proposal by adding a copy layer and a dynamic knowledge base vocabulary to two Seq2Seq architectures (CNNs and Transformers). This layer makes the models copy KB elements directly from the questions, instead of generating them. We evaluate our approach on state-of-the-art datasets, including datasets referencing unknown KB elements and measure the accuracy of the copy-augmented architectures. Our results show a considerable increase in performance on all datasets compared to non-copy architectures.