# Lexicalization Is All You Need: Examining the Impact of Lexical Knowledge in a Compositional QALD System

arXiv: https://arxiv.org/abs/2411.03906 · HTML: https://mlsanity.com/paper/2411.03906/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 81 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases](https://mlsanity.com/paper/2011.07743/) (2020)
- [Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text](https://mlsanity.com/paper/1809.00782/) (2018)
- [Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family](https://mlsanity.com/paper/2303.07992/) (2023)
- [Question Answering with Subgraph Embeddings](https://mlsanity.com/paper/1406.3676/) (2014)
- [Leveraging Abstract Meaning Representation for Knowledge Base Question Answering](https://mlsanity.com/paper/2012.01707/) (2020)
- [ArcaneQA: Dynamic Program Induction and Contextualized Encoding for Knowledge Base Question Answering](https://mlsanity.com/paper/2204.08109/) (2022)
- [ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbots](https://mlsanity.com/paper/2302.06466/) (2023)
- [Neural Machine Translating from Natural Language to SPARQL](https://mlsanity.com/paper/1906.09302/) (2019)
- [A Universal Question-Answering Platform for Knowledge Graphs](https://mlsanity.com/paper/2303.00595/) (2023)
- [Towards a Question Answering System over the Semantic Web](https://mlsanity.com/paper/1803.00832/) (2018)
- [Generative Relation Linking for Question Answering over Knowledge Bases](https://mlsanity.com/paper/2108.07337/) (2021)
- [Pretrained Transformers for Simple Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2001.11985/) (2020)
- [ForecastTKGQuestions: A Benchmark for Temporal Question Answering and Forecasting over Temporal Knowledge Graphs](https://mlsanity.com/paper/2208.06501/) (2022)
- [Knowledge Graph Question Answering using Graph-Pattern Isomorphism](https://mlsanity.com/paper/2103.06752/) (2021)
- [Question Answering over Knowledge Graphs via Structural Query Patterns](https://mlsanity.com/paper/1910.09760/) (2019)
- [Generating Questions for Knowledge Bases via Incorporating Diversified Contexts and Answer-Aware Loss](https://mlsanity.com/paper/1910.13108/) (2019)
- [Leveraging Semantic Parsing for Relation Linking over Knowledge Bases](https://mlsanity.com/paper/2009.07726/) (2020)
- [Template-based Question Answering using Recursive Neural Networks](https://mlsanity.com/paper/2004.13843/) (2020)
- [QAGCN: Answering Multi-Relation Questions via Single-Step Implicit Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2206.01818/) (2022)
- [Question Answering on Linked Data: Challenges and Future Directions](https://mlsanity.com/paper/1601.03541/) (2016)
- [AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data](https://mlsanity.com/paper/1802.09296/) (2018)

## Built on by (most cited first, 1 of 1)

- [CompoST: A Benchmark for Analyzing the Ability of LLMs To Compositionally Interpret Questions in a QALD Setting](https://mlsanity.com/paper/2507.21257/) (2025)

## Abstract (arXiv)

In this paper, we examine the impact of lexicalization on Question Answering over Linked Data (QALD). It is well known that one of the key challenges in interpreting natural language questions with respect to SPARQL lies in bridging the lexical gap, that is mapping the words in the query to the correct vocabulary elements. We argue in this paper that lexicalization, that is explicit knowledge about the potential interpretations of a word with respect to the given vocabulary, significantly eases the task and increases the performance of QA systems. Towards this goal, we present a compositional QA system that can leverage explicit lexical knowledge in a compositional manner to infer the meaning of a question in terms of a SPARQL query. We show that such a system, given lexical knowledge, has a performance well beyond current QA systems, achieving up to a 35.8% increase in the micro F1 score compared to the best QA system on QALD-9. This shows the importance and potential of including explicit lexical knowledge. In contrast, we show that LLMs have limited abilities to exploit lexical knowledge, with only marginal improvements compared to a version without lexical knowledge. This shows that LLMs have no ability to compositionally interpret a question on the basis of the meaning of its parts, a key feature of compositional approaches. Taken together, our work shows new avenues for QALD research, emphasizing the importance of lexicalization and compositionality.