mlsanityData to 7 Oct 2026

Library Learning Doesn't: The Curious Case of the Single-Use "Library"

arXiv 2410.20274 ↗First posted 26 Oct 2024cs.LG cs.CL cs.SCcited by 3

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Advances in Large Language Models (LLMs) have spurred a wave of LLM library learning systems for mathematical reasoning. These systems aim to learn a reusable library of tools, such as formal Isabelle lemmas or Python programs that are tailored to a family of tasks. Many of these systems are inspired by the human structuring of knowledge into reusable and extendable concepts, but do current methods actually learn reusable libraries of tools? We study two library learning systems for mathematics which both reported increased accuracy: LEGO-Prover and TroVE. We find that function reuse is extremely infrequent on miniF2F and MATH. Our followup ablation experiments suggest that, rather than reuse, self-correction and self-consistency are the primary drivers of the observed performance gains. Our code and data are available at https://github.com/ikb-a/curious-case

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 36 references that are in the map. First posted on arXiv 26 Oct 2024.

Builds on

30 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Machine learning and information theory concepts towards an AI Mathematician7 Mar

Other references · 29

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  8. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  9. 2023Large Language Models as Tool Makerscited by 293
  10. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  11. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  12. 2024Tool Learning with Large Language Models: A Surveycited by 221
17 more
  1. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  2. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Modelscited by 129
  3. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  4. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 122
  5. 2023LEGO-Prover: Neural Theorem Proving with Growing Librariescited by 108
  6. 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Modelscited by 90
  7. 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
  8. 2024A Survey on Deep Learning for Theorem Provingcited by 55
  9. 2024Offline Training of Language Model Agents with Functions as Learnable Weightscited by 47
  10. 2023LILO: Learning Interpretable Libraries by Compressing and Documenting Codecited by 46
  11. 2024ReGAL: Refactoring Programs to Discover Generalizable Abstractionscited by 32
  12. 2024Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generationcited by 27
  13. 2024Empowering Large Language Model Agents through Action Learningcited by 20
  14. 2023GENOME: GenerativE Neuro-symbOlic visual reasoning by growing and reusing ModulEscited by 18
  15. 2024REFACTOR: Learning to Extract Theorems from Proofscited by 9
  16. 2023Can LLM find the green circle? Investigation and Human-guided tool manipulation for compositional generalizationcited by 7
  17. 2024ATG: Benchmarking Automated Theorem Generation for Generative Language Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025LLM Library Learning Fails: A LEGO-Prover Case Studycited by 3
  2. 2025A Compute-Matched Re-Evaluation of TroVE on MATHcited by 2
  3. 2026AllocBench: Measuring Online Tool Allocation Capability in LLM Agentscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Thinking Machines: Mathematical Reasoning in the Age of LLMs
  2. 2025DRIFT: Decompose, Retrieve, Illustrate, then Formalize Theorems
  3. 2025Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics
  4. 2026MathlibLemma: Folklore Lemma Generation and Benchmark for Formal Mathematics
  5. 2026Less Is More: Cognitive Load and the Single-Prompt Ceiling in LLM Mathematical Reasoning
  6. 2025Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities
  7. 2025Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models
  8. 2026Beyond the Library: An Agentic Framework for Autoformalizing Research Mathematics