mlsanityData to 14 Oct 2026

Maestro Order: A Model-Agnostic Orchestration Harness

arXiv 2606.23983 ↗First posted 22 Jun 2026cs.CR cs.AI cs.LOcited by 0

Not on the map. Why?

Abstract (arXiv)

A single forward pass of a capable model is a fast, fluent, and unreliable problem-solver: it is right often enough to be useful and wrong often enough to be dangerous; in language models, such confident errors are known as hallucinations. We present Maestro Order, a model-agnostic orchestration harness that turns unreliable solvers into reliable problem-solving systems by composing them according to four structural primitives (decompose, ensemble, verify, and recurse) and a budget-aware controller that decides where to spend compute. The harness treats any model as a black-box base solver behind a uniform interface, layers a verifier ensemble whose discrimination is measured online, and allocates verification and voting to the stages with the highest marginal reliability per unit cost. We give the architecture, the message and state schema, the controller algorithm, and the engineering that makes it deterministic, observable, and fault-tolerant. We then specify an evaluation methodology (reliability at fixed cost, coverage, calibration, and ablations) and report results from a faithful Monte Carlo simulation of the harness over a parameterized solver/verifier model. The simulation reproduces the predicted laws quantitatively: verification amplifies reliability geometrically (e.g. 0.55→0.98 with two gates, →0.999 with four), voting helps only above chance and is limited by shared errors, and a budget-aware controller reaches a target reliability at a small fraction of the cost of voting alone by selecting the cheapest mechanism for each regime. We close with failure modes (verifier gaming, correlated errors, and decomposition error compounding) and concrete guidance: build robust checkers, diversify solvers, and let the controller put compute where the information is.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 17 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

10 of 17 references.

References · 10

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  10. 2026Odds Law: The Decomposition Algebra On How Intelligence Organizes Itself to Solve Difficult Problems Reliablycited by 2

Built on by

Not cited yet.