mlsanityData to 14 Oct 2026

Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning

arXiv 2610.13152 ↗First posted 9 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents also fits: General LLM agent frameworks and benchmarks

Abstract (arXiv)

Open coding agents are commonly trained through supervised fine-tuning on interaction trajectories whose final patches pass the benchmark test suite. However, this outcome-level filtering criterion ignores process quality: trajectories that eventually succeed may still contain inefficient interaction patterns such as repetitive file inspection, unstable tool usage, or delayed modification behavior, all of which become part of the imitation target. We propose TQS (Trajectory Quality Score), a structural scoring framework for coding-agent trajectories that operates directly on trajectory structure, requires no LLM judge, and runs in linear time. TQS characterizes trajectory quality through structural signals organized around structured convergence: information focus, intent--action alignment, and policy dynamics. Using 155,144 successful trajectories from a public open-license corpus, we rank trajectories by TQS and fine-tune coding agents on filtered subsets. Models trained on top-ranked trajectories consistently outperform those trained on random or bottom-ranked subsets on SWE-Bench-Verified. Our results suggest that process-level trajectory quality is an important factor in coding-agent supervision and that structural trajectory filtering provides an effective and scalable alternative to LLM-based data selection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 35 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

22 of 35 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents21 May
  2. 2026Beyond Quantity: Trajectory Diversity Scaling for Code Agents3 Feb
  3. 2025SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents26 May
  4. 2025SWE-smith: Scaling Data for Software Engineering Agents30 Apr
  5. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents9 Apr
  6. 2025OrcaLoca: An LLM Agent Framework for Software Issue Localization1 Feb
  7. 2024Training Software Engineering Agents and Verifiers with SWE-Gym30 Dec
  8. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement1 Nov
  9. 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
  10. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 12

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2023LIMA: Less Is More for Alignmentcited by 1,100
  4. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  5. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  6. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  7. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 263
  8. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  9. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 163
  10. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  11. 2024Chain-of-Thought Unfaithfulness as Disguised Accuracycited by 28
  12. 2025Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimizationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents
  2. 2026TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training
  3. 2026Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models
  4. 2026Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework
  5. 2026Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories
  6. 2026What Process Evaluation of Coding Agents Actually Measures: Action, Task, and Step Are Three Different Levels
  7. 2026DynSTEER: Dynamic Stage-wise Trajectory Evaluation and Execution-time Review for Agents
  8. 2026Opera: A Verbal Critic Framework for Long-horizon Coding Agents