mlsanityData to 14 Oct 2026

SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

arXiv 2609.30054 ↗First posted 24 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Improving the scientific coding capabilities of large language models (LLMs) requires high-quality training data. However, such data remain scarce because manually authoring realistic problems is costly and time-consuming, while systematically covering diverse scientific domains and algorithmic combinations remains challenging. To address this, we introduce SciWalker, a framework for synthesizing scientific coding problems through operator-chain sampling and execution feedback. The framework combines scientific library interfaces with operation modes to instantiate operators, organizes them into operator graphs, and samples operator chains as computational workflow cues. Guided by these cues, we adopt LLMs to generate scientifically grounded problem statements, reference solutions, and tests, with failed generations iteratively repaired using execution feedback. By combining structured workflow composition with verification and quality review, SciWalker enables scalable task generation while promoting scientific grounding, computational diversity, and executability. Using this framework, we construct 8,178 high-quality problems spanning 5 scientific domains and 32 subdomains. To evaluate their training utility, we conduct reinforcement learning on Qwen3.5-9B using the GSPO algorithm. This training improves SciCode subproblem accuracy by 9.9 percentage points, from 29.3% to 39.2%, with gains across scientific code generation, code repair, and reasoning benchmarks. The code for SciWalker is available at https://github.com/lichenx1/SciWalker.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 20 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

16 of 20 references.

References · 16

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  8. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  9. 2021Measuring Coding Challenge Competence With APPScited by 902
  10. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  11. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  12. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
4 more
  1. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  2. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
  3. 2024SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Modelscited by 39
  4. 2026WildSci: Advancing Scientific Reasoning from In-the-Wild Literaturecited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SciNav: A General Agent Framework for Scientific Coding Tasks
  2. 2025MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding
  3. 2025QueST: Incentivizing LLMs to Generate Difficult Problems
  4. 2026No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows
  5. 2025Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
  6. 2025AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
  7. 2025RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback
  8. 2026Combinatorial Synthesis: Scaling Code RLVR via Atomic Decomposition and Recombination