# Language Models for Business Optimisation with a Real World Case Study in Production Scheduling

arXiv: https://arxiv.org/abs/2309.13218 · HTML: https://mlsanity.com/paper/2309.13218/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 78 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning](https://mlsanity.com/paper/2207.01780/) (2022)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Planning with Large Language Models for Code Generation](https://mlsanity.com/paper/2303.05510/) (2023)
- [Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets](https://mlsanity.com/paper/2106.10328/) (2021)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [Progressive-Hint Prompting Improves Reasoning in Large Language Models](https://mlsanity.com/paper/2304.09797/) (2023)
- [Large Language Models Meet NL2Code: A Survey](https://mlsanity.com/paper/2212.09420/) (2022)
- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [Evolution through Large Models](https://mlsanity.com/paper/2206.08896/) (2022)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering](https://mlsanity.com/paper/1908.08167/) (2019)
- [Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT](https://mlsanity.com/paper/2304.10778/) (2023)
- [Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks](https://mlsanity.com/paper/2102.02017/) (2021)
- [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023)
- [Augmenting Operations Research with Auto-Formulation of Optimization Models from Problem Descriptions](https://mlsanity.com/paper/2209.15565/) (2022)
- [ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing](https://mlsanity.com/paper/2303.13367/) (2023)
- [OPD@NL4Opt: An ensemble approach for the NER task of the optimization problem](https://mlsanity.com/paper/2301.02459/) (2023)

## Built on by (most cited first, 4 of 4)

- [From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto](https://mlsanity.com/paper/2402.16269/) (2024)
- [GreedLlama: Performance of Financial Value-Aligned Large Language Models in Moral Reasoning](https://mlsanity.com/paper/2404.02934/) (2024)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization](https://mlsanity.com/paper/2508.18091/) (2025)

## Abstract (arXiv)

Business optimisation has been used extensively to determine optimal solutions for challenging business operations. Problem formulation is an important part of business optimisation as it influences both the validity of solutions and the efficiency of the optimisation process. While different optimisation modelling languages have been developed, problem formulation is still not a trivial task and usually requires optimisation expertise and problem-domain knowledge. Recently, Large Language Models (LLMs) have demonstrated outstanding performance across different language-related tasks. Since problem formulation can be viewed as a translation task, there is a potential to leverage LLMs to automate problem formulation. However, developing an LLM for problem formulation is challenging, due to limited training data, and the complexity of real-world optimisation problems. Several prompt engineering methods have been proposed in the literature to automate problem formulation with LLMs. While the initial results are encouraging, the accuracy of formulations generated by these methods can still be significantly improved. In this paper, we present an LLM-based framework for automating problem formulation in business optimization. Our approach introduces a method for fine-tuning cost-efficient LLMs specifically tailored to specialized business optimization challenges. The experiment results demonstrate that our framework can generate accurate formulations for conventional and real-world business optimisation problems in production scheduling. Extensive analyses show the effectiveness and the convergence of the proposed fine-tuning method. The proposed method also shows very competitive performance when compared with the state-of-the-art prompt engineering methods in the literature when tested on general linear programming problems.