The Conversations Beneath the Code: Triadic Data for Long-Horizon Software Engineering Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Frontier software engineering agents have saturated short-horizon benchmarks while regressing on the work that constitutes senior engineering: long-horizon, multi-engineer, ambiguous-specification deliverables. This paper takes a position on what training data is needed to close the gap. The substrate for the next generation of SWE agents is neither larger GitHub scrapes nor more solo-agent trajectories nor -- sufficient by itself -- open human-AI dialogue logs. It is triadic data: synchronized capture of the human-human conversations where engineering context is formed, the human-AI sessions where that context is partially consumed, and the multi-week cross-functional work that surrounds both. We argue that the canonical instantiation of triadic data is two complementary products: long-horizon expert trajectories captured under stimulated-recall protocols, and simulated cross-functional companies -- instrumented teams of senior engineers, product managers, designers, and data scientists working through ambiguous deliverables on shared infrastructure. We further specify a four-tier evidence framework through which any such corpus -- triadic or otherwise -- must justify its quality to a fine-tuning researcher: mechanical verification, statistical corpus characterization, probe experiments, and pre-registered blind evaluation. We argue that this data is capturable in 12-18 months with methods already mature in adjacent fields, that it is the empirical key to four open questions in agent training, and that the field's near-term research agenda should include it explicitly.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 20 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
11 of 20 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2023LIMA: Less Is More for Alignmentcited by 1,100
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
- 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 203
- 2022Do Users Write More Insecure Code with AI Assistants?cited by 192
- 2025Measuring AI Ability to Complete Long Software Taskscited by 156
- 2025SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?cited by 105
- 2024SWE-Bench+: Enhanced Coding Benchmark for LLMscited by 99
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025From Correctness to Collaboration: Toward a Human-Centered Framework for Evaluating AI Agent Behavior in Software Engineering
- 2026Human-AI Collaboration and the Transformation of Software Engineering Work
- 2026Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents
- 2026Agentic AI in the Software Development Lifecycle: Architecture, Empirical Evidence, and the Reshaping of Software Engineering
- 2025The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
- 2026Projecting the Emerging Mindset of SWE Agent by Launching a Wild Code Understanding Journey
- 2026Rethinking Software Engineering for Agentic AI Systems
- 2026From Determinism to Delegation: AI-Native Software Engineering and the Evolution of the Agentic Engineer