mlsanityData to 14 Oct 2026

Three-Phase Evaluation of AI-Assisted Software Development Life Cycle

arXiv 2607.05125 ↗First posted 6 Jul 2026cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents an exploratory evaluation of how increasing levels of AI autonomy affect software development productivity, requirement adherence, and developer cognitive workload. A team of four developers reimplemented the same full-stack web application across three sequential phases: partial AI-assisted development using GitHub Copilot, an AI-exclusive workflow using GitHub Copilot, and an AI-exclusive workflow using AWS Kiro. Evaluation metrics included development effort (hours), requirement adherence (RITM score), AI-interaction efficiency, and NASA-TLX workload measures. Across phases, higher levels of AI autonomy were associated with reduced development effort, improved requirement adherence, and lower self-reported mental workload, while developer frustration increased modestly. The AWS Kiro phase achieved the strongest overall performance on most measured dimensions, suggesting that tooling architecture may influence outcomes independently of AI autonomy level.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 43 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

21 of 43 references.

References · 21

  1. 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 271
  2. 2022SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebookscited by 125
  3. 2024BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometrycited by 103
  4. 2021BRepNet: A topological message passing system for solid modelscited by 95
  5. 2023Hierarchical Neural Coding for Controllable CAD Model Generationcited by 86
  6. 2024CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLMcited by 81
  7. 2021JoinABLe: Learning Bottom-up Assembly of Parametric CAD Jointscited by 78
  8. 2024CAD-Recode: Reverse Engineering CAD Code from Point Cloudscited by 66
  9. 2025CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Rewardcited by 56
  10. 2024CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMscited by 53
  11. 2025Text-to-CadQuery: A New Paradigm for CAD Generation with Scalable Large Model Capabilitiescited by 43
  12. 2024CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solverscited by 40
9 more
  1. 2023Point2CAD: Reverse Engineering CAD Models from 3D Point Cloudscited by 36
  2. 2025CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generationcited by 19
  3. 2023ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibilitycited by 18
  4. 2026CADSmith: Multi-Agent CAD Generation with Programmatic Geometric Validationcited by 13
  5. 2025A Solver-Aided Hierarchical Language for LLM-Driven CAD Designcited by 10
  6. 2026IterCAD: An Iterative Multimodal Agent for Visually-Grounded CAD Generation and Editingcited by 10
  7. 2022Mates2Motion: Learning How Mechanical CAD Assemblies Workcited by 2
  8. 2026ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigmcited by 2
  9. 2022A review on geometric constraint solvingcited by 1

Built on by

Not cited yet.