# Project SHADOW: Symbolic Higher-order Associative Deductive reasoning On Wikidata using LM probing

arXiv: https://arxiv.org/abs/2408.14849 · HTML: https://mlsanity.com/paper/2408.14849/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 10 of its 11 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 10 of 10)

- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [Probing Pretrained Language Models for Lexical Semantics](https://mlsanity.com/paper/2010.05731/) (2020)
- [Prompting as Probing: Using Language Models for Knowledge Base Construction](https://mlsanity.com/paper/2208.11057/) (2022)
- [Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs](https://mlsanity.com/paper/2408.00114/) (2024)
- [Can Language Models Act as Knowledge Bases at Scale?](https://mlsanity.com/paper/2402.14273/) (2024)
- [Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs](https://mlsanity.com/paper/2404.17120/) (2024)
- [Language Modeling with Editable External Knowledge](https://mlsanity.com/paper/2406.11830/) (2024)
- [Large Language Model Enhanced Knowledge Representation Learning: A Survey](https://mlsanity.com/paper/2407.00936/) (2024)
- [Probing Commonsense Knowledge in Pre-trained Language Models with Sense-level Precision and Expanded Vocabulary](https://mlsanity.com/paper/2210.06376/) (2022)

## Abstract (arXiv)

We introduce SHADOW, a fine-tuned language model trained on an intermediate task using associative deductive reasoning, and measure its performance on a knowledge base construction task using Wikidata triple completion. We evaluate SHADOW on the LM-KBC 2024 challenge and show that it outperforms the baseline solution by 20% with a F1 score of 68.72%.