# Agility in Software 2.0 -- Notebook Interfaces and MLOps with Buttresses and Rebars

arXiv: https://arxiv.org/abs/2111.14142 · HTML: https://mlsanity.com/paper/2111.14142/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 47 references that are in the map.
- First posted on arXiv 28 Nov 2021.

## Builds on (most cited first, 11 of 11)

- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [Requirements Engineering for Machine Learning: Perspectives from Data Scientists](https://mlsanity.com/paper/1908.04674/) (2019)
- [Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry](https://mlsanity.com/paper/1812.05389/) (2018)
- [Engineering AI Systems: A Research Agenda](https://mlsanity.com/paper/2001.07522/) (2020)
- [A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges](https://mlsanity.com/paper/2012.07919/) (2020)
- [Verification for Machine Learning, Autonomy, and Neural Networks Survey](https://mlsanity.com/paper/1810.01989/) (2018)
- [Non-functional Requirements for Machine Learning: Understanding Current Use and Challenges in Industry](https://mlsanity.com/paper/2109.00872/) (2021)
- [Certification of embedded systems based on Machine Learning: A survey](https://mlsanity.com/paper/2106.07221/) (2021)
- [Test Automation with Grad-CAM Heatmaps -- A Future Pipe Segment in MLOps for Vision AI?](https://mlsanity.com/paper/2103.01837/) (2021)

## Built on by (most cited first, 4 of 4)

- [MLOps: A Review](https://mlsanity.com/paper/2308.10908/) (2023)
- [Exploring ML testing in practice -- Lessons learned from an interactive rapid review with Axis Communications](https://mlsanity.com/paper/2203.16225/) (2022)
- [Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice](https://mlsanity.com/paper/2203.15414/) (2022)
- [A Systematic Literature Review of Soft Computing Techniques for Software Maintainability Prediction: State-of-the-Art, Challenges and Future Directions](https://mlsanity.com/paper/2209.10131/) (2022)

## Abstract (arXiv)

Artificial intelligence through machine learning is increasingly used in the digital society. Solutions based on machine learning bring both great opportunities, thus coined "Software 2.0," but also great challenges for the engineering community to tackle. Due to the experimental approach used by data scientists when developing machine learning models, agility is an essential characteristic. In this keynote address, we discuss two contemporary development phenomena that are fundamental in machine learning development, i.e., notebook interfaces and MLOps. First, we present a solution that can remedy some of the intrinsic weaknesses of working in notebooks by supporting easy transitions to integrated development environments. Second, we propose reinforced engineering of AI systems by introducing metaphorical buttresses and rebars in the MLOps context. Machine learning-based solutions are dynamic in nature, and we argue that reinforced continuous engineering is required to quality assure the trustworthy AI systems of tomorrow.