mlsanityData to 7 Oct 2026

XDomainBench: Diagnosing Reasoning Collapse in High-Dimensional Scientific Knowledge Composition

arXiv 2605.14754 ↗First posted 14 May 2026cs.AIcited by 1

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed for knowledge synthesis, yet their capacity for compositional generalization in scientific knowledge remains under-characterized. Existing benchmarks primarily focus on single-turn restricted scenarios, failing to capture the capability boundaries exposed by real-world interactive scientific workflows. To address this, we introduce XDomainBench, a diagnostic benchmark for interactive interdisciplinary scientific reasoning. We formalize the composition order and mixture structure to enable systematic stress-testing from single-discipline to inter-disciplinary, comprising 8,598 interactive sessions across 20 domains and 4 task categories, with 8 realistic trajectory patterns covering difficulty and domain-mixture dynamics, simulating real AI4S scenarios. Large-scale evaluation of LLMs reveals a systematic reasoning collapse as composition order increases, stemming from two root causes: (i) direct difficulty increases induced by domain composition, and (ii) indirect interaction-amplified failures where trajectory patterns trigger error accumulation, reasoning breaks, and domain confusion, ultimately leading to session collapse.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 46 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

30 of 46 references.

References · 30

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  10. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  11. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
18 more
  1. 2024Mixtral of Expertscited by 1,785
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  3. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  5. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  9. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  10. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  11. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  12. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
  13. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  14. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  15. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
  16. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  17. 2020Domain Divergences: a Survey and Empirical Analysiscited by 24
  18. 2025MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrievalcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs
  2. 2026SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence
  3. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
  4. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
  5. 2026ProjectionBench: Evaluating Scientific Hypothesis Generation in LLMs Under Progressive Information Disclosure
  6. 2026SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
  7. 2026X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes
  8. 2026SciR: A Controllable Benchmark for Scientific Reasoning in LLMs