# StepGap: A Hybrid NLI-LLM Checker for Step-Level Evidence-Gap Detectionin Multi-Hop Question Answering

arXiv: https://arxiv.org/abs/2605.24733 · HTML: https://mlsanity.com/paper/2605.24733/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 23 May 2026.

## Builds on (most cited first, 17 of 17)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [MMSearch-R1: Incentivizing LMMs to Search](https://mlsanity.com/paper/2506.20670/) (2025)
- [TRAIL: Trace Reasoning and Agentic Issue Localization](https://mlsanity.com/paper/2505.08638/) (2025)
- [ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding](https://mlsanity.com/paper/2501.07861/) (2025)
- [RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism](https://mlsanity.com/paper/2507.02962/) (2025)
- [Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Experts](https://mlsanity.com/paper/2502.08825/) (2025)
- [DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradients](https://mlsanity.com/paper/2505.19538/) (2025)

## Abstract (arXiv)

We present StepGap, a hybrid NLI-LLM decision tree that detects step-level evidence gaps in multi-hop QA and emits one of three typed labels: textscContradicted Claim (CC), textscIrrelevant Evidence (IE), or textscMissing Bridge (MB), each tied to a concrete repair action. On 82 multi-hop questions (181 annotated steps, kappa=0.704), StepGap reaches sF1=72.0, within the bootstrap confidence interval of an LLM-only baseline (70.1) but with a more decomposable structure: every StepGap stage hurts F1 when removed, while three of four LLM-only removals improve F1 -- a sign of competing-error cancellation, where internal stages mask each other's errors. We further expose a Q-F1 trap: question-level F1 is mechanically inflated by checkers that flag every step, making step-level F1 the necessary diagnostic. Used as a typed GRPO process reward, StepGap improves Qwen2.5-7B-Instruct Exact Match from 32.1±0.3 to 35.4±0.9 across three seeds, with the single-run comparison showing a +5.6 Avg EM gain over the matched Search-R1 GRPO reproduction.