mlsanityData to 7 Oct 2026

JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution

arXiv 2608.25593 ↗First posted 26 Aug 2026cs.CL cs.LGcited by 10

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliable execution, and self-evolve by distilling performance signals from an expanding archive of prior harness configurations. Equipped with JIT-Agent as a harness helper, DeepSeek-V4-Flash surpasses GPT-5.6 on DeepSearchQA (+9.1) and OdysseyBench (+4.3), while the already strong GLM-5.2 gains up to +20.2 points. Across controlled evaluations, JIT-Agent-generated harnesses are performance-competitive with mature agent runtimes such as OpenCode and Claude Code and consistently improve multi-scale model families of DeepSeek V4, Mimo-V2.5, and Qwen3.6. To our knowledge, JIT-Agent is the first model purpose-built for just-in-time harness generation, establishing harness intelligence as a trainable, transferable, and compounding dimension of agent capability orthogonal to model scaling.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 60 of its 84 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

60 of 84 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories3 Aug
  2. 2026HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses3 Aug
  3. 2026Self-Harness: Harnesses That Improve Themselves8 Jun
  4. 2026Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams1 Jun
  5. 2026Code as Agent Harness18 May
  6. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
  7. 2024AFlow: Automating Agentic Workflow Generation14 Oct
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 52

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  4. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  5. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  6. 2024Automated Design of Agentic Systemscited by 360
  7. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  8. 2025Memory in the Age of AI Agentscited by 285
  9. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
  10. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 193
  11. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  12. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 183
38 more
  1. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 175
  2. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
  3. 2025Multi-agent Architecture Search via Agentic Supernetcited by 164
  4. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
  5. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 132
  6. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  7. 2024EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithmscited by 109
  8. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
  9. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 101
  10. 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 95
  11. 2025Recursive Language Modelscited by 94
  12. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  13. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  14. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  15. 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scalecited by 65
  16. 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 63
  17. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 62
  18. 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 54
  19. 2026Natural-Language Agent Harnessescited by 54
  20. 2026EverMemOS: A Self-Organizing Memory Operating System for Structured Long-Horizon Reasoningcited by 52
  21. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
  22. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 44
  23. 2026DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraintscited by 43
  24. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  25. 2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundrycited by 37
  26. 2026Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agentscited by 37
  27. 2026AOrchestra: Automating Sub-Agent Creation for Agentic Orchestrationcited by 30
  28. 2025Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Executioncited by 27
  29. 2026Recursive Harness Self-Improvementcited by 22
  30. 2026Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Taskscited by 18
  31. 2026Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preferencecited by 17
  32. 2025Survey of LLM Agent Communication with MCP: A Software Design Pattern Centric Reviewcited by 14
  33. 2026MemoBrain: Executive Memory as an Agentic Brain for Reasoningcited by 12
  34. 2026ClawGym: A Scalable Framework for Building Effective Claw Agentscited by 11
  35. 2025HiRA: A Hierarchical Reasoning Framework for Decoupled Planning and Execution in Deep Searchcited by 8
  36. 2026AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarioscited by 8
  37. 2026Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnessescited by 4
  38. 2026ROMA: Recursive Open Meta-Agent Framework for Long-Horizon Multi-Agent Systemscited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Harness-Zero: Harness Distillation via Agent-as-Harnesscited by 3
  2. 2026OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learningcited by 2
  3. 2026More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnessescited by 1
  4. 2026Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memorycited by 0
  5. 2026JET: Judge-Guided Evolution at Test Time for Agent Programscited by 0
  6. 2026Harness Learning Enables Generalizable Test-Time Adaptationcited by 0
  7. 2026Composing Task-specific Agent Harnesses at Test Time with Reusable Primitivescited by 0
  8. 2026Turbo Harness: Instance-Adaptive Harness Optimizationcited by 0
  9. 2026EvoGen-Harness: Learning Where and How to Evolve Image-Generation Harnessescited by 0
  10. 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LiteEvo: Automated, Cost-Efficient Harness Evolution for Generalization to Unseen Tasks
  2. 2026HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
  3. 2026Rethinking the Evaluation of Harness Evolution for Agents
  4. 2026HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
  5. 2026EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
  6. 2026The Interplay of Harness Design and Post-Training in LLM Agents
  7. 2026Dynamic Harness Search: Building Multi-Agent Systems Per-Query via Prediction
  8. 2026EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness?