# Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices

arXiv: https://arxiv.org/abs/1703.07019 · HTML: https://mlsanity.com/paper/1703.07019/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2017.

## Built on by (most cited first, 22 of 22)

- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Towards Autonomous Testing Agents via Conversational Large Language Models](https://mlsanity.com/paper/2306.05152/) (2023)
- [Systematic Literature Review on Application of Learning-based Approaches in Continuous Integration](https://mlsanity.com/paper/2406.19765/) (2024)
- [SoK: Machine Learning for Continuous Integration](https://mlsanity.com/paper/2304.02829/) (2023)
- [DialogVCS: Robust Natural Language Understanding in Dialogue System Upgrade](https://mlsanity.com/paper/2305.14751/) (2023)
- [Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications](https://mlsanity.com/paper/2603.15676/) (2026)
- [The End of Code Review: Coding Agents Supersede Human Inspection](https://mlsanity.com/paper/2606.13175/) (2026)
- [Systematic Literature Review on Application of Machine Learning in Continuous Integration](https://mlsanity.com/paper/2305.12695/) (2023)
- [Towards a Middleware for Large Language Models](https://mlsanity.com/paper/2411.14513/) (2024)
- [Graph-Based Model-Agnostic Data Subsampling for Recommendation Systems](https://mlsanity.com/paper/2305.16391/) (2023)
- [Leveraging Reward Models for Guiding Code Review Comment Generation](https://mlsanity.com/paper/2506.04464/) (2025)
- [Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights](https://mlsanity.com/paper/2509.05142/) (2025)
- [Towards Knowledge Organization Ecosystems](https://mlsanity.com/paper/2105.10923/) (2021)
- [Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks](https://mlsanity.com/paper/2503.17630/) (2025)
- [Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap](https://mlsanity.com/paper/2505.19625/) (2025)
- [A Generative Approach for Semantic Auditing of Electronic Health Records](https://mlsanity.com/paper/2507.02628/) (2025)
- [A Concept for Efficient Scalability of Automated Driving Allowing for Technical, Legal, Cultural, and Ethical Differences](https://mlsanity.com/paper/2507.18326/) (2025)
- [The Alignment Flywheel: A Governance-Centric Hybrid MAS for Architecture-Agnostic Safety](https://mlsanity.com/paper/2603.02259/) (2026)
- [CCCE: A Continuous Code Calibration Engine for Autonomous Enterprise Codebase Maintenance via Knowledge Graph Traversal and Adaptive Decision Gating](https://mlsanity.com/paper/2604.13102/) (2026)
- [From Assistance to Agency: Rethinking Autonomy and Control in CI/CD Pipelines](https://mlsanity.com/paper/2605.07062/) (2026)
- [Latent Anchor-Driven Test Generation for Deep Neural Networks](https://mlsanity.com/paper/2606.04310/) (2026)
- [LLM-Driven CI-CD Workflow Intelligence for Cyber Systems Engineering](https://mlsanity.com/paper/2607.04579/) (2026)

## Abstract (arXiv)

Context: Continuous practices, i.e., continuous integration, delivery, and deployment, are the software development industry practices that enable organizations to frequently and reliably release new features and products. With the increasing interest in and literature on continuous practices, it is important to systematically review and synthesize the approaches, tools, challenges, and practices reported for adopting and implementing continuous practices. Objective: This research aimed at systematically reviewing the state of the art of continuous practices to classify approaches and tools, identify challenges and practices in this regard, and identify the gaps for future research. Method: We used systematic literature review (SLR) method for reviewing the peer-reviewed papers on continuous practices published between 2004 and 1st June 2016. We applied thematic analysis method for analysing the data extracted from reviewing 69 papers selected using predefined criteria. Results: We have identified thirty approaches and associated tools, which facilitate the implementation of continuous practices in the following ways: (1) "reducing build and test time in continuous integration (CI)"; (2) "increasing visibility and awareness on build and test results in CI"; (3) "supporting (semi-) automated continuous testing"; (4) "detecting violations, flaws and faults in CI"; (5) "addressing security and scalability issues in deployment pipeline", and (6) "improving dependability and reliability of deployment process". We have also determined a list of critical factors such as "testing (effort and time)", "team awareness and transparency", "good design principles", "customer", "highly skilled and motivated team", "application domain", and "appropriate infrastructure" that should be carefully considered when introducing continuous practices in a given organization.