# How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?

arXiv: https://arxiv.org/abs/2609.40303 · HTML: https://mlsanity.com/paper/2609.40303/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 25 of 25)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [Rethinking the Evaluation of Harness Evolution for Agents](https://mlsanity.com/paper/2607.12227/) (2026)
- [R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science](https://mlsanity.com/paper/2505.14738/) (2025)
- [MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery](https://mlsanity.com/paper/2606.06473/) (2026)
- [MARS: Modular Agent with Reflective Search for Automated AI Research](https://mlsanity.com/paper/2602.02660/) (2026)
- [MLZero: A Multi-Agent System for End-to-end Machine Learning Automation](https://mlsanity.com/paper/2505.13941/) (2025)
- [Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline](https://mlsanity.com/paper/2601.12307/) (2026)
- [Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent](https://mlsanity.com/paper/2606.30616/) (2026)
- [ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence](https://mlsanity.com/paper/2605.26340/) (2026)
- [AutoMLGen: Navigating Fine-Grained Optimization for Coding Agents](https://mlsanity.com/paper/2510.08511/) (2025)
- [Toward Autonomous Long-Horizon Engineering for ML Research](https://mlsanity.com/paper/2604.13018/) (2026)
- [KAPSO: A Knowledge-grounded framework for Autonomous Program Synthesis and Optimization](https://mlsanity.com/paper/2601.21526/) (2026)
- [Synthetic Sandbox for Training Machine Learning Engineering Agents](https://mlsanity.com/paper/2604.04872/) (2026)
- [KompeteAI: Accelerated Autonomous Multi-Agent System for End-to-End Pipeline Generation for Machine Learning Problems](https://mlsanity.com/paper/2508.10177/) (2025)
- [Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering](https://mlsanity.com/paper/2607.28568/) (2026)
- [Learning to Ideate for Machine Learning Engineering Agents](https://mlsanity.com/paper/2601.17596/) (2026)
- [Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search](https://mlsanity.com/paper/2603.01692/) (2026)
- [IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agents](https://mlsanity.com/paper/2605.22154/) (2026)
- [Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineering](https://mlsanity.com/paper/2607.25090/) (2026)
- [Beyond Solution-Centric Search: Adaptive Inquiry and Knowledge Revision for Autonomous ML Engineering](https://mlsanity.com/paper/2608.02143/) (2026)

## Abstract (arXiv)

Recent autonomous machine learning engineering (MLE) agents have made significant progress on public leaderboards. Often motivated by progress stagnation over long-horizon cycles and limited Large Language Model (LLM) primitives, modern MLE agents are deployed on top of increasingly elaborate machinery: multi-agent orchestrators, dedicated retrieval subagents, and more. While such harnesses expand, the use of more primitive but improved coding agents - where LLMs have direct access to the execution environment through read, write, and bash primitives - has received little attention in the field. In this paper we find that, under an equal time budget and the same frontier LLM backbone, open-source state-of-the-art harnesses provide no advantages over a single session of a minimal-harness coding agent baseline, pointing to the backbone as the primary driver for performance. Via a series of large-scale systematic ablation studies, we argue that the machinery layers become redundant in the coding agent setting. We conclude that the effort spent elaborating hand-crafted harnesses around strong models yields poor returns for current MLE benchmarks.