StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?
Not on the map. Why?
Abstract (arXiv)
Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolution method should share the same property, that is autonomously learning from raw training material for transferable problem-solving capability. However, we still lack a direct measurement for it. We introduce StudyBench, a controlled physics benchmark that directly measures how efficiently a self-evolution method converts training material into capability. We organise the test set into an Application Set, consisting of difficult textbook problems and evaluating absorption ability, and a Transfer Set, consisting of olympiad-level problems and evaluating transfer ability. Benchmarking representative self-evolution methods across three base models, we find that improvements on the Application Set rarely translate to the harder Transfer Set. A guidance ablation exposes a Guidance Gap: even the strongest method closes only a small fraction of what the same material unlocks when supplied as in-context guidance. Besides, every method hits a Compute Plateau, saturating well before exhausting its compute budget. The remaining gap is therefore a method problem rather than a data or compute problem. By offering a clean and controlled benchmark, StudyBench turns self-evolution progress from an open-ended pursuit into a measurable target for future research. Our code is released at https://github.com/thunlp/StudyBench.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
14 of 34 references.
References · 14
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
- 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
- 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
- 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
- 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 78
- 2025NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agentscited by 36
- 2024Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptationcited by 27
- 2025StoryBench: A Dynamic Benchmark for Evaluating Long-Term Memory with Multi Turnscited by 20
- 2026Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimizationcited by 16
- 2025EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solvingcited by 10
Built on by
Not cited yet.