mlsanityData to 14 Oct 2026

STREET: A Multi-Task Structured Reasoning and Explanation Benchmark

arXiv 2302.06729 ↗First posted 13 Feb 2023cs.CL cs.AIcited by 13

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We introduce STREET, a unified multi-task and multi-domain natural language reasoning and explanation benchmark. Unlike most existing question-answering (QA) datasets, we expect models to not only answer questions, but also produce step-by-step structured explanations describing how premises in the question are used to produce intermediate conclusions that can prove the correctness of a certain answer. We perform extensive evaluation with popular language models such as few-shot prompting GPT-3 and fine-tuned T5. We find that these models still lag behind human performance when producing such structured reasoning steps. We believe this work will provide a way for the community to better train and test systems on multi-step reasoning and explanations in natural language.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 38 of its 51 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

38 of 51 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,847
26 more
  1. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  2. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  3. 2019Attention is not Explanationcited by 929
  4. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  6. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  7. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  8. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  9. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  10. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  11. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  12. 2020Evaluation of Text Generation: A Surveycited by 277
  13. 2020Transformers as Soft Reasoners over Languagecited by 270
  14. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  15. 2020Open Question Answering over Tables and Textcited by 163
  16. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 162
  17. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  18. 2021Explaining Answers with Entailment Treescited by 152
  19. 2018WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-Hop Inferencecited by 60
  20. 2021AR-LSAT: Investigating Analytical Reasoning of Textcited by 51
  21. 2020Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answeringcited by 49
  22. 2016Simpler Context-Dependent Logical Forms via Model Projectionscited by 33
  23. 2020Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?cited by 30
  24. 2022Entailment Tree Explanations via Iterative Retrieval-Generation Reasonercited by 26
  25. 2020Generating Rationales in Visual Question Answeringcited by 6
  26. 2019A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks Worldcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 7; 2024: 5; 2025: 1.

13 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  3. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  4. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  5. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 119
  6. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 102
  7. 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correctioncited by 45
  8. 2024LOGIC-LM++: Multi-Step Refinement for Symbolic Formulationscited by 14
  9. 2024Evaluating LLM Reasoning in the Operations Research Domain with ORQAcited by 12
  10. 2023Resprompt: Residual Connection Prompting Advances Multi-Step Reasoning in Large Language Modelscited by 7
  11. 2024Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thoughtcited by 4
  12. 2024SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learningcited by 3
1 more
  1. 2025MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality
  2. 2026From Chains to Graphs: Self-Structured Reasoning for General-Domain LLMs
  3. 2024Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification
  4. 2025Evaluating the Meta- and Object-Level Reasoning of Large Language Models for Question Answering
  5. 2025ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models
  6. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge
  7. 2026Exploring the Meta-level Reasoning of Large Language Models via a Tool-based Multi-hop Tabular Question Answering Task
  8. 2024FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models