mlsanityData to 14 Oct 2026

VLM Fine-Tuning for End-to-End Combinatorial Optimization

arXiv 2609.37175 ↗First posted 29 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have provided a unified interface for end-to-end combinatorial optimization (CO), but textual serialization alone may obscure spatial and relational structures that are important for generating effective CO solutions. This paper presents a general-purpose vision-language solver that augments textual instance descriptions with input-derived visual representations. A single vision-language model (VLM) is applied across different CO tasks and trained using supervised fine-tuning followed by verifier-guided reinforcement learning. While the visual inputs contain no gold solutions or solution-derived information, our experiments show that the VLM generally improves solution quality over its text-only counterpart, with particularly clear gains on more complex CO problems such as CVRP and JSSP. The advantage of visual information is more pronounced at large problem scales.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

12 of 19 references.

References · 12

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Large Language Models as Optimizerscited by 864
  3. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 276
  4. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  5. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  6. 2023Large Language Models as Evolutionary Optimizerscited by 124
  7. 2024LLMOPT: Learning to Define and Solve General Optimization Problems from Scratchcited by 102
  8. 2023RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmarkcited by 71
  9. 2025Large Language Models as End-to-end Combinatorial Optimization Solverscited by 28
  10. 2024Self-Guiding Exploration for Combinatorial Problemscited by 22
  11. 2023Job Shop Scheduling Benchmark: Environments and Instances for Learning and Non-learning Methodscited by 12
  12. 2026Hard Constraints Meet Soft Generation: Guaranteed Feasibility for LLM-based Combinatorial Optimizationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Bridging Large Language Models and Optimization: A Unified Framework for Text-attributed Combinatorial Optimization
  2. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  3. 2025VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making
  4. 2026DRAGON: LLM-Driven Decomposition and Reconstruction Agents for Large-Scale Combinatorial Optimization
  5. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning
  6. 2025GFlowVLM: Enhancing Multi-step Reasoning in Vision-Language Models with Generative Flow Networks
  7. 2024Exploring Combinatorial Problem Solving with Large Language Models: A Case Study on the Travelling Salesman Problem Using GPT-3.5 Turbo
  8. 2026COVR:Collaborative Optimization of VLMs and RL Agent for Visual-Based Control