mlsanityData to 14 Oct 2026

REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs

arXiv 2608.10963 ↗First posted 11 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present the REAP system for the AKBC Shared Task 2026 on constructing knowledge bases from language models in a closed-book setting, subject to a budget of at most 32B parameters and no model fine-tuning. Our system combines structured chain-of-thought reasoning, relation-specific query strategies, and a reasoning-based empty-set gate to elicit parametric knowledge, followed by direct extraction into valid JSON arrays. On the test set, the system, built on the Mistral-Small-24B-Instruct-2501 model, achieves a macro-F1 score of 0.62, with particularly strong results on countryLandBordersCountry (F1 = 0.95), companyTradesAtStockExchange (F1 = 0.73), and hasArea (F1 = 0.77). Our code is publicly available at https://github.com/yammdd/AKBC-Shared-Task-2026.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 17 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

11 of 17 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2019Language Models as Knowledge Bases?cited by 1,936
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  6. 2019How Can We Know What Language Models Know?cited by 959
  7. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  8. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  9. 2020Selective Question Answering under Domain Shiftcited by 238
  10. 2022A Review on Language Models as Knowledge Basescited by 157
  11. 2023Using Large Language Models for Knowledge Engineering (LLMKE): A Case Study on Wikidatacited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning of Large Language Models over Knowledge Graphs with Super-Relations
  2. 2026ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models
  3. 2026RPO-RAG: Aligning Small LLMs with Relation-aware Preference Optimization for Knowledge Graph Question Answering
  4. 2025An LLM + ASP Workflow for Joint Entity-Relation Extraction
  5. 2025Structure-Augmented Reasoning Generation
  6. 2026Evaluating Relational Reasoning in LLMs with REL
  7. 2023Pay More Attention to Relation Exploration for Knowledge Base Question Answering
  8. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models