mlsanityData to 14 Oct 2026

OR-Toolformer: Modeling and Solving Operations Research Problems with Tool Augmented Large Language Models

arXiv 2510.01253 ↗First posted 24 Sep 2025cs.AI cs.LGcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large language models (LLMs) demonstrate strong mathematical reasoning, but reliance on closed-source APIs for OR tasks raises privacy concerns, and training open-source models from scratch incurs high compute costs. We introduce OR-Toolformer, which fine-tunes Llama-3.1-8B-Instruct with a semi-automatic data synthesis pipeline that generates diverse OR problem-answer pairs and augments the model with external solvers to produce API calls. On three of four standard benchmarks, OR-Toolformer achieves up to 80.1% execution accuracy, exceeding size-matched baselines by over 4.3%. In zero-shot evaluation on two unseen OR problem types, it attains 54% average accuracy, a 21 percentage-point improvement over the strongest baseline. These findings validate the efficacy of tool-augmented fine-tuning LLMs for accurate and generalizable OR problem modeling and solving.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 28 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

18 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Evaluating LLM Reasoning in the Operations Research Domain with ORQA22 Dec

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  5. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  6. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  7. 2024A Survey on Data Selection for Language Modelscited by 236
  8. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 204
  9. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  10. 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 135
  11. 2024ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modelingcited by 124
  12. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 82
5 more
  1. 2024Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agentscited by 56
  2. 2024Learning to Use Tools via Cooperative and Interactive Agentscited by 54
  3. 2024Understanding the Performance and Estimating the Cost of LLM Fine-Tuningcited by 48
  4. 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learningcited by 47
  5. 2024Budget-Constrained Tool Learning with Planningcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
  2. 2024LLM With Tools: A Survey
  3. 2022TALM: Tool Augmented Language Models
  4. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets
  5. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
  6. 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models
  7. 2026Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives
  8. 2023Large Language Models as Tool Makers