mlsanityData to 14 Oct 2026

StepGap: A Hybrid NLI-LLM Checker for Step-Level Evidence-Gap Detectionin Multi-Hop Question Answering

arXiv 2605.24733 ↗First posted 23 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present StepGap, a hybrid NLI-LLM decision tree that detects step-level evidence gaps in multi-hop QA and emits one of three typed labels: textscContradicted Claim (CC), textscIrrelevant Evidence (IE), or textscMissing Bridge (MB), each tied to a concrete repair action. On 82 multi-hop questions (181 annotated steps, kappa=0.704), StepGap reaches sF1=72.0, within the bootstrap confidence interval of an LLM-only baseline (70.1) but with a more decomposable structure: every StepGap stage hurts F1 when removed, while three of four LLM-only removals improve F1 -- a sign of competing-error cancellation, where internal stages mask each other's errors. We further expose a Q-F1 trap: question-level F1 is mechanically inflated by checkers that flag every step, making step-level F1 the necessary diagnostic. Used as a typed GRPO process reward, StepGap improves Qwen2.5-7B-Instruct Exact Match from 32.1±0.3 to 35.4±0.9 across three seeds, with the single-run comparison showing a +5.6 Avg EM gain over the matched Search-R1 GRPO reproduction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

17 of 31 references.

References · 17

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  5. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  7. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  8. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  9. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  10. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  11. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  12. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
5 more
  1. 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 104
  2. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
  3. 2025RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelismcited by 16
  4. 2025Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Expertscited by 9
  5. 2025DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradientscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Hi-Q: Hierarchical Evidence-guided Query Refinement for Multi-Hop Question Answering
  2. 2026SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning
  3. 2026Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering
  4. 2026AdaGATE: Adaptive Gap-Aware Token-Efficient Evidence Assembly for Multi-Hop Retrieval-Augmented Generation
  5. 2025PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering
  6. 2026Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
  7. 2026REALHOP: Rethinking Multi-Hop Reasoning Evaluation via Behavioral Auditing
  8. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge