Project SHADOW: Symbolic Higher-order Associative Deductive reasoning On Wikidata using LM probing
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
We introduce SHADOW, a fine-tuned language model trained on an intermediate task using associative deductive reasoning, and measure its performance on a knowledge base construction task using Wikidata triple completion. We evaluate SHADOW on the LM-KBC 2024 challenge and show that it outperforms the baseline solution by 20% with a F1 score of 68.72%.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 11 references that are in the map. First posted on arXiv 27 Aug 2024.
Builds on
10 of 11 references.
Closest ancestors · 2
Same direction, newest first
- 2024Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs31 Jul
- 2022Probing Commonsense Knowledge in Pre-trained Language Models with Sense-level Precision and Expanded Vocabulary12 Oct
Other references · 8
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2020Probing Pretrained Language Models for Lexical Semanticscited by 156
- 2022A Review on Language Models as Knowledge Basescited by 156
- 2022Prompting as Probing: Using Language Models for Knowledge Base Constructioncited by 32
- 2024Can Language Models Act as Knowledge Bases at Scale?cited by 17
- 2024Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputscited by 11
- 2024Language Modeling with Editable External Knowledgecited by 9
- 2024Large Language Model Enhanced Knowledge Representation Learning: A Surveycited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024The Role of Deductive and Inductive Reasoning in Large Language Models
- 2023Learning Deductive Reasoning from Synthetic Corpus based on Formal Logic
- 2023Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming
- 2025SR-FoT: A Syllogistic-Reasoning Framework of Thought for Large Language Models Tackling Knowledge-based Reasoning Tasks
- 2022Evaluating Step-by-Step Reasoning through Symbolic Verification
- 2024Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoning
- 2024FLARE: Faithful Logic-Aided Reasoning and Exploration
- 2025Investigating the Robustness of Deductive Reasoning with Large Language Models