Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
Not on the map. Why?
Abstract (arXiv)
Small open-weight models (2-9B) run on ordinary laptops, but under cloud-scale agent harnesses they rarely complete real tasks: tool prefill overflows the context, self-correction diverges, tool demonstrations loop, and tasks are silently abandoned. We present evidence, from a controlled single-machine comparison and one third-party benchmark, that a substantial share of these failures is attributable to the harness rather than the model. We introduce Mingbird, a local-first agent harness for Windows and Ollama whose ten mechanisms compensate point-by-point for small-model failure forms, three of them representative: a byte-level net-zero prefill budget, a finish gate that re-reads the task before accepting completion, and signature-level loop detection. On LRAB, a controlled comparison holding machine, models, budgets, and scoring fixed (4 harnesses × 4 open models (2B-35B) × 18 real tasks, deterministic artifact scoring), Mingbird reaches 0.886 overall against 0.631 (goose), 0.479 (opencode), and 0.405 (agent-mini), with all 288 cells published; on τ2-bench (278 tasks, three arms, one protocol) it totals 0.856 against 0.791 and 0.737; and a frontier-model probe on the same 18 tasks spans 0.997 to 0.478 across harnesses, with well-formed scaffolds staying within 0.072 of each other. A leave-one-mechanism-out ablation is reported as directional only: same-night replications of the same arm move its mean by up to 0.069, the size of every nominal single-trial delta, and the one batch-matched comparison (full mechanism stack versus text re-read alone) gives the executable completion guards a paired +0.10 across three replications. The evidence carries stated limits: a self-built benchmark, a single machine, and single-trial scoring.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 60 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
46 of 60 references.
References · 46
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen3 Technical Reportcited by 8,195
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
34 more
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
- 2024AFlow: Automating Agentic Workflow Generationcited by 395
- 2024Automated Design of Agentic Systemscited by 360
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
- 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
- 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
- 2025Defeating Prompt Injections by Designcited by 256
- 2025Small Language Models are the Future of Agentic AIcited by 254
- 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
- 2024AI Agents That Mattercited by 199
- 2023Efficient Guided Generation for Large Language Modelscited by 192
- 2026Gemma 4 Technical Reportcited by 187
- 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
- 2024Small Language Models: Survey, Measurements, and Insightscited by 109
- 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 99
- 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
- 2024Language Agents as Optimizable Graphscited by 81
- 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 73
- 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 69
- 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 66
- 2024TinyAgent: Function Calling at the Edgecited by 55
- 2026Stop Comparing LLM Agents Without Disclosing the Harnesscited by 25
- 2026Scaffold Effects on GAIA: A Controlled Comparisoncited by 3
Built on by
Not cited yet.