Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Terminal agents act through stochastic model generations, yet the ability to generate a useful action does not ensure its reliable execution. A poor command (e.g., wrong package install) can change the environment in ways that hinder subsequent progress, even when the model could generate a better alternative. We investigate whether allocating test-time compute at the model-harness boundary can improve action reliability and trajectory success, and what makes this allocation effective. To study these questions, we introduce Mid-Harness, which samples and verifies candidate actions before forwarding one for execution, while keeping the generator and harness unchanged. With a TMAX-9B generator, more action sampling yields little benefit under weak verification, whereas a capable verifier can exploit useful alternatives from the same generator. On TerminalBench-Lite, a GPT-5.6 Sol verifier raises Pass@1 from 50.00% for the base agent to 68.03% with 8 sampled actions. When the same TMAX-9B model serves as the verifier, pairwise verification performs best among the evaluated verification mechanisms. Distilling responses from the stronger verifier into TMAX-9B further improves Pass@1, while leaving the action generator unchanged. With TMAX-9B on TerminalBench-Lite, combining action and trajectory scaling reaches higher success at lower estimated token cost than generating more trajectories alone. Mid-Harness also improves performance across additional models, benchmarks, and harnesses. These findings identify action scaling as a promising target for test-time compute scaling in terminal agents.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 81 of its 108 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
81 of 108 references.
Closest ancestors · 10
Same direction, newest first
- 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesis7 Sep
- 2026Prime Agent: A Self-Improving RLM Harness24 Aug
- 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Development11 Feb
- 2026EGSS: Entropy-guided Stepwise Scaling for Reliable Software Engineering5 Feb
- 2026Agentic Rubrics as Contextual Verifiers for SWE Agents7 Jan
- 2025SWE-RM: Execution-free Feedback For Software Engineering Agents26 Dec
- 2025When Agents go Astray: Course-Correcting SWE Agents with PRMs2 Sep
- 2025Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents19 May
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
Other references · 71
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Let's Verify Step by Stepcited by 3,598
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2025s1: Simple test-time scalingcited by 1,311
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
- 2023Reasoning with Language Model is Planning with World Modelcited by 846
- 2016Sequence-Level Knowledge Distillationcited by 761
38 more
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
- 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
- 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
- 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
- 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 126
- 2025Process Reward Models That Thinkcited by 118
- 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 72
- 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
- 2025Scaling Test-time Compute for LLM Agentscited by 59
- 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
- 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
- 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
- 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 41
- 2024MoAI: Mixture of All Intelligence for Large Language and Vision Modelscited by 36
- 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 34
- 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
- 2022Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneckcited by 30
- 2024Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Modelscited by 30
- 2024CoLLaVO: Crayon Large Language and Vision mOdelcited by 26
- 2026Tmax: A simple recipe for terminal agentscited by 24
- 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
- 2022Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Networkcited by 21
- 2022KALA: Knowledge-Augmented Language Model Adaptationcited by 21
- 2023Knowledge-Augmented Language Model Verificationcited by 21
- 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 21
- 2025Rethinking Thinking Tokens: LLMs as Improvement Operatorscited by 20
- 2023Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regressioncited by 18
- 2023Causal Unsupervised Semantic Segmentationcited by 18
- 2024Phantom of Latent for Large Language and Vision Modelscited by 15
- 2019Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Datacited by 14
- 2025GenSelect: A Generative Approach to Best-of-Ncited by 14
- 2024TroL: Traversal of Layers for Large Language and Vision Modelscited by 13
- 2025T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Modelscited by 13
- 2025RefineBench: Evaluating Refinement Capability of Language Models via Checklistscited by 13
- 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 13
- 2026V1: Unifying Generation and Self-Verification for Parallel Reasonerscited by 12
- 2023Mitigating Adversarial Vulnerability through Causal Parameter Estimation by Adversarial Double Machine Learningcited by 11
50 most cited of 71 All 71 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TTHE: Test-Time Harness Evolution
- 2026Harnesses for Inference-Time Alignment over Execution Trajectories
- 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
- 2026Stop Comparing LLM Agents Without Disclosing the Harness
- 2026Self-Harness: Harnesses That Improve Themselves
- 2026Look Before You Leap: Pre-Action Verification for LLM Agents
- 2026Harness-Zero: Harness Distillation via Agent-as-Harness
- 2026Rethinking the Evaluation of Harness Evolution for Agents