mlsanityData to 7 Oct 2026

Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents

arXiv 2609.39982 ↗First posted 30 Sep 2026cs.CL cs.AI cs.MAcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Terminal agents act through stochastic model generations, yet the ability to generate a useful action does not ensure its reliable execution. A poor command (e.g., wrong package install) can change the environment in ways that hinder subsequent progress, even when the model could generate a better alternative. We investigate whether allocating test-time compute at the model-harness boundary can improve action reliability and trajectory success, and what makes this allocation effective. To study these questions, we introduce Mid-Harness, which samples and verifies candidate actions before forwarding one for execution, while keeping the generator and harness unchanged. With a TMAX-9B generator, more action sampling yields little benefit under weak verification, whereas a capable verifier can exploit useful alternatives from the same generator. On TerminalBench-Lite, a GPT-5.6 Sol verifier raises Pass@1 from 50.00% for the base agent to 68.03% with 8 sampled actions. When the same TMAX-9B model serves as the verifier, pairwise verification performs best among the evaluated verification mechanisms. Distilling responses from the stronger verifier into TMAX-9B further improves Pass@1, while leaving the action generator unchanged. With TMAX-9B on TerminalBench-Lite, combining action and trajectory scaling reaches higher success at lower estimated token cost than generating more trajectories alone. Mid-Harness also improves performance across additional models, benchmarks, and harnesses. These findings identify action scaling as a promising target for test-time compute scaling in terminal agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 81 of its 108 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

81 of 108 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesis7 Sep
  2. 2026Prime Agent: A Self-Improving RLM Harness24 Aug
  3. 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Development11 Feb
  4. 2026EGSS: Entropy-guided Stepwise Scaling for Reliable Software Engineering5 Feb
  5. 2026Agentic Rubrics as Contextual Verifiers for SWE Agents7 Jan
  6. 2025SWE-RM: Execution-free Feedback For Software Engineering Agents26 Dec
  7. 2025When Agents go Astray: Course-Correcting SWE Agents with PRMs2 Sep
  8. 2025Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents19 May
  9. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
  10. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 71

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2025s1: Simple test-time scalingcited by 1,311
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2016Sequence-Level Knowledge Distillationcited by 761
38 more
  1. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  2. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  3. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
  4. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
  5. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
  6. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 126
  7. 2025Process Reward Models That Thinkcited by 118
  8. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 72
  9. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
  10. 2025Scaling Test-time Compute for LLM Agentscited by 59
  11. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
  12. 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
  13. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  14. 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 41
  15. 2024MoAI: Mixture of All Intelligence for Large Language and Vision Modelscited by 36
  16. 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 34
  17. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
  18. 2022Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneckcited by 30
  19. 2024Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Modelscited by 30
  20. 2024CoLLaVO: Crayon Large Language and Vision mOdelcited by 26
  21. 2026Tmax: A simple recipe for terminal agentscited by 24
  22. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  23. 2022Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Networkcited by 21
  24. 2022KALA: Knowledge-Augmented Language Model Adaptationcited by 21
  25. 2023Knowledge-Augmented Language Model Verificationcited by 21
  26. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 21
  27. 2025Rethinking Thinking Tokens: LLMs as Improvement Operatorscited by 20
  28. 2023Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regressioncited by 18
  29. 2023Causal Unsupervised Semantic Segmentationcited by 18
  30. 2024Phantom of Latent for Large Language and Vision Modelscited by 15
  31. 2019Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Datacited by 14
  32. 2025GenSelect: A Generative Approach to Best-of-Ncited by 14
  33. 2024TroL: Traversal of Layers for Large Language and Vision Modelscited by 13
  34. 2025T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Modelscited by 13
  35. 2025RefineBench: Evaluating Refinement Capability of Language Models via Checklistscited by 13
  36. 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 13
  37. 2026V1: Unifying Generation and Self-Verification for Parallel Reasonerscited by 12
  38. 2023Mitigating Adversarial Vulnerability through Causal Parameter Estimation by Adversarial Double Machine Learningcited by 11

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TTHE: Test-Time Harness Evolution
  2. 2026Harnesses for Inference-Time Alignment over Execution Trajectories
  3. 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
  4. 2026Stop Comparing LLM Agents Without Disclosing the Harness
  5. 2026Self-Harness: Harnesses That Improve Themselves
  6. 2026Look Before You Leap: Pre-Action Verification for LLM Agents
  7. 2026Harness-Zero: Harness Distillation via Agent-as-Harness
  8. 2026Rethinking the Evaluation of Harness Evolution for Agents