mlsanityData to 7 Oct 2026

Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning

arXiv 2607.22621 ↗First posted 17 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

While large language models (LLMs) enable strong question answering (QA), budgeted deployment is complicated by nondeterminism and heterogeneous resource profiles (cost, latency, and energy). We present OPTI-Q, a database-inspired, cost-based optimizer that implements a plan-before-execute paradigm for multi-LLM orchestration. OPTI-Q models LLM invocations as physical operators in an execution DAG and, for each question, searches for plans that optimize answer quality (QoA) while trading off financial cost, latency, and energy under user-specified resource constraints. Plans can include sequential operators that pass intermediate answers as context and parallel/blend operators that run models concurrently and merge their outputs. To search this space without executing each candidate plan, OPTI-Q uses PERFDB, a statistics catalog populated and refreshed from benchmarks and execution traces, to estimate the QoA and resource costs of both individual operators and composed subplans. Using these estimates, OPTI-Q performs Pareto-frontier search and selects a final plan based on user preferences. On MMLU-Pro and SimpleQA under user-specified budgets, OPTI-Q improves average QoA by ~58% and ~41% over baselines at comparable cost, demonstrating that database-style planning yields better quality-resource trade-offs for multi-LLM QA.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

39 of 53 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2023Mistral 7Bcited by 3,065
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  11. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  12. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
27 more
  1. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  2. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2024Phi-4 Technical Reportcited by 675
  5. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  6. 2024Measuring short-form factuality in large language modelscited by 345
  7. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
  8. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
  9. 2024The rising costs of training frontier AI modelscited by 91
  10. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  11. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  12. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 78
  13. 2020Bao: Learning to Steer Query Optimizerscited by 62
  14. 2024Prompt Design and Engineering: Introduction and Advanced Methodscited by 45
  15. 2024LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversitycited by 42
  16. 2024Towards Optimizing the Costs of LLM Usagecited by 33
  17. 2021Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluationcited by 30
  18. 2023Lero: A Learning-to-Rank Query Optimizercited by 26
  19. 2025When One LLM Drools, Multi-LLM Collaboration Rulescited by 25
  20. 2025ThriftLLM: On Cost-Effective Selection of Large Language Models for Classification Queriescited by 14
  21. 2023Querying Large Language Models with SQLcited by 13
  22. 2025Uncertainty-Aware Fusion: An Ensemble Framework for Mitigating Hallucinations in Large Language Modelscited by 12
  23. 2025Abacus: A Cost-Based Optimizer for Semantic Operator Systemscited by 11
  24. 2023GenAIPABench: A Benchmark for Generative AI-based Privacy Assistantscited by 7
  25. 2025A Query Optimization Method Utilizing Large Language Modelscited by 3
  26. 2026The Stretto Execution Engine for LLM-Augmented Data Systemscited by 2
  27. 2016A Fast Randomized Algorithm for Multi-Objective Query Optimizationcited by 1

Built on by

Not cited yet.