# Three-Phase Evaluation of AI-Assisted Software Development Life Cycle

arXiv: https://arxiv.org/abs/2607.05125 · HTML: https://mlsanity.com/paper/2607.05125/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 21 of 21)

- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry](https://mlsanity.com/paper/2401.15563/) (2024)
- [BRepNet: A topological message passing system for solid models](https://mlsanity.com/paper/2104.00706/) (2021)
- [Hierarchical Neural Coding for Controllable CAD Model Generation](https://mlsanity.com/paper/2307.00149/) (2023)
- [CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM](https://mlsanity.com/paper/2411.04954/) (2024)
- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [CAD-Recode: Reverse Engineering CAD Code from Point Clouds](https://mlsanity.com/paper/2412.14042/) (2024)
- [CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward](https://mlsanity.com/paper/2505.19713/) (2025)
- [CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs](https://mlsanity.com/paper/2412.19663/) (2024)
- [Text-to-CadQuery: A New Paradigm for CAD Generation with Scalable Large Model Capabilities](https://mlsanity.com/paper/2505.06507/) (2025)
- [CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers](https://mlsanity.com/paper/2412.13810/) (2024)
- [Point2CAD: Reverse Engineering CAD Models from 3D Point Clouds](https://mlsanity.com/paper/2312.04962/) (2023)
- [CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation](https://mlsanity.com/paper/2512.23333/) (2025)
- [ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility](https://mlsanity.com/paper/2309.16909/) (2023)
- [CADSmith: Multi-Agent CAD Generation with Programmatic Geometric Validation](https://mlsanity.com/paper/2603.26512/) (2026)
- [A Solver-Aided Hierarchical Language for LLM-Driven CAD Design](https://mlsanity.com/paper/2502.09819/) (2025)
- [IterCAD: An Iterative Multimodal Agent for Visually-Grounded CAD Generation and Editing](https://mlsanity.com/paper/2606.13368/) (2026)
- [Mates2Motion: Learning How Mechanical CAD Assemblies Work](https://mlsanity.com/paper/2208.01779/) (2022)
- [ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm](https://mlsanity.com/paper/2606.13239/) (2026)
- [A review on geometric constraint solving](https://mlsanity.com/paper/2202.13795/) (2022)

## Abstract (arXiv)

This paper presents an exploratory evaluation of how increasing levels of AI autonomy affect software development productivity, requirement adherence, and developer cognitive workload. A team of four developers reimplemented the same full-stack web application across three sequential phases: partial AI-assisted development using GitHub Copilot, an AI-exclusive workflow using GitHub Copilot, and an AI-exclusive workflow using AWS Kiro. Evaluation metrics included development effort (hours), requirement adherence (RITM score), AI-interaction efficiency, and NASA-TLX workload measures. Across phases, higher levels of AI autonomy were associated with reduced development effort, improved requirement adherence, and lower self-reported mental workload, while developer frustration increased modestly. The AWS Kiro phase achieved the strongest overall performance on most measured dimensions, suggesting that tooling architecture may influence outcomes independently of AI autonomy level.