# GRS-QA -- Graph Reasoning-Structured Question Answering Dataset

arXiv: https://arxiv.org/abs/2411.00369 · HTML: https://mlsanity.com/paper/2411.00369/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements](https://mlsanity.com/paper/2402.10963/) (2024)
- [SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs](https://mlsanity.com/paper/2404.13081/) (2024)
- [Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2309.11206/) (2023)
- [Dynamic Planning with a LLM](https://mlsanity.com/paper/2308.06391/) (2023)
- [LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning](https://mlsanity.com/paper/2401.00125/) (2023)
- [Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning](https://mlsanity.com/paper/2405.20625/) (2024)
- [Modeling Multi-hop Question Answering as Single Sequence Prediction](https://mlsanity.com/paper/2205.09226/) (2022)

## Built on by (most cited first, 4 of 4)

- [Enhancing Agentic Textual Graph Retrieval with Synthetic Stepwise Supervision](https://mlsanity.com/paper/2510.03323/) (2025)
- [Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzles](https://mlsanity.com/paper/2511.06160/) (2025)
- [Improving LLMs with a knowledge from databases](https://mlsanity.com/paper/2506.05560/) (2025)
- [Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)](https://mlsanity.com/paper/2606.05901/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have excelled in multi-hop question-answering (M-QA) due to their advanced reasoning abilities. However, the impact of the inherent reasoning structures on LLM M-QA performance remains unclear, largely due to the absence of QA datasets that provide fine-grained reasoning structures. To address this gap, we introduce the Graph Reasoning-Structured Question Answering Dataset (GRS-QA), which includes both semantic contexts and reasoning structures for QA pairs. Unlike existing M-QA datasets, where different reasoning structures are entangled together, GRS-QA explicitly captures intricate reasoning pathways by constructing reasoning graphs, where nodes represent textual contexts and edges denote logical flows. These reasoning graphs of different structures enable a fine-grained evaluation of LLM reasoning capabilities across various reasoning structures. Our empirical analysis reveals that LLMs perform differently when handling questions with varying reasoning structures. This finding facilitates the exploration of textual structures as compared with semantics.