mlsanityData to 7 Oct 2026

Software Engineering for AI-Based Systems: A Survey

arXiv 2105.01984 ↗First posted 5 May 2021cs.SE cs.AI cs.LGcited by 68

Not on the map. Why?

Abstract (arXiv)

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 83 of its 236 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

83 of 236 references.

References · 83

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  4. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  5. 2016Safety Verification of Deep Neural Networkscited by 262
  6. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  7. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 249
  8. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 213
  9. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
  10. 2018Formal Security Analysis of Neural Networks using Symbolic Intervalscited by 185
  11. 2018Efficient Formal Safety Analysis of Neural Networkscited by 165
  12. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 129
38 more
  1. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  2. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  3. 2018Automated Directed Fairness Testingcited by 77
  4. 2017Robust Physical-World Attacks on Deep Learning Modelscited by 70
  5. 2018TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzingcited by 67
  6. 2019A Comprehensive Study on Deep Learning Bug Characteristicscited by 65
  7. 2016On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscited by 52
  8. 2018Concolic Testing for Deep Neural Networkscited by 51
  9. 2018DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systemscited by 50
  10. 2018Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failurecited by 49
  11. 2019Requirements Engineering for Machine Learning: Perspectives from Data Scientistscited by 49
  12. 2018On Testing Machine Learning Programscited by 48
  13. 2018Model-Reuse Attacks on Deep Learning Systemscited by 47
  14. 2017Machine Teaching: A New Paradigm for Building Machine Learning Systemscited by 41
  15. 2018Software Engineering Challenges of Deep Learningcited by 41
  16. 2019Towards Corner Case Detection for Autonomous Drivingcited by 39
  17. 2018Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industrycited by 38
  18. 2020Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real Worldcited by 38
  19. 2020Engineering AI Systems: A Research Agendacited by 37
  20. 2016Towards Verified Artificial Intelligencecited by 36
  21. 2019An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platformscited by 35
  22. 2020Adoption and Effects of Software Engineering Best Practices in Machine Learningcited by 33
  23. 2020A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challengescited by 33
  24. 2018FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformitycited by 31
  25. 2018Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testingcited by 30
  26. 2019DeepFault: Fault Localization for Deep Neural Networkscited by 29
  27. 2017An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Softwarecited by 26
  28. 2016Engineering Safety in Machine Learningcited by 24
  29. 2018SOTER: A Runtime Assurance Framework for Programming Safe Robotics Systemscited by 24
  30. 2018Applying Deep Learning To Airbnb Searchcited by 24
  31. 2019Studying Software Engineering Patterns for Designing Machine Learning Systemscited by 24
  32. 2019Requirements Engineering Challenges in Building AI-Based Complex Systemscited by 21
  33. 2017DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networkscited by 20
  34. 2019Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Componentscited by 17
  35. 2018Semantic Adversarial Deep Learningcited by 15
  36. 2020Software engineering for artificial intelligence and machine learning software: A systematic literature reviewcited by 15
  37. 2018Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262cited by 13
  38. 2019What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflowcited by 13

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 11; 2023: 15; 2024: 20; 2025: 13; 2026: 7.

68 papers.

Sort:

By area:

  1. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 65
  2. 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
  3. 2023Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Studycited by 23
  4. 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
  5. 2023Analyzing the Evolution and Maintenance of ML Models on Hugging Facecited by 22
  6. 2023METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualitiescited by 20
  7. 2022Bugs in Machine Learning-based Systems: A Faultload Benchmarkcited by 15
  8. 2023The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practicecited by 14
  9. 2023Bug Characterization in Machine Learning-based Systemscited by 10
  10. 2023Data Cleaning and Machine Learning: A Systematic Literature Reviewcited by 9
  11. 2022Machine Learning for Software Engineering: A Tertiary Studycited by 7
  12. 2024A Survey on Failure Analysis and Fault Injection in AI Systemscited by 7
38 more
  1. 2021Which Design Decisions in AI-enabled Mobile Applications Contribute to Greener AI?cited by 6
  2. 2022Towards a Change Taxonomy for Machine Learning Systemscited by 6
  3. 2023Can ChatGPT advance software testing intelligence? An experience report on metamorphic testingcited by 6
  4. 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systemscited by 6
  5. 2024Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Modelscited by 6
  6. 2024How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directionscited by 5
  7. 2022Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systemscited by 4
  8. 2022A Compositional Approach to Creating Architecture Frameworks with an Application to Distributed AI Systemscited by 4
  9. 2023Design Patterns for AI-based Systems: A Multivocal Literature Review and Pattern Repositorycited by 4
  10. 2023User-Like Bots for Cognitive Automation: A Surveycited by 4
  11. 2024Deep Learning Library Testing: Definition, Methods and Challengescited by 4
  12. 2024A Catalog of Fairness-Aware Practices in Machine Learning Engineeringcited by 4
  13. 2025Leveraging LLMs for User Stories in AI Systems: UStAI Datasetcited by 4
  14. 2022Differential testing for machine learning: an analysis for classification algorithms beyond deep learningcited by 3
  15. 2022Leveraging Artificial Intelligence on Binary Code Comprehensioncited by 3
  16. 2023Towards green AI-based software systems: an architecture-centric approach (GAISSA)cited by 3
  17. 2024How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detectioncited by 3
  18. 2024Naming the Pain in Machine Learning-Enabled Systems Engineeringcited by 3
  19. 2024Compliance Cards: Automated EU AI Act Compliance Analyses amidst a Complex AI Supply Chaincited by 3
  20. 2024Generating a Low-code Complete Workflow via Task Decomposition and RAGcited by 3
  21. 2025Engineering LLM Powered Multi-agent Framework for Autonomous CloudOpscited by 3
  22. 2026From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agentscited by 3
  23. 2022Modeling Quality and Machine Learning Pipelines through Extended Feature Modelscited by 2
  24. 2023Bridging MDE and AI: A Systematic Review of Domain-Specific Languages and Model-Driven Practices in AI Software Systems Engineeringcited by 2
  25. 2024A Conceptual Framework for Ethical Evaluation of Machine Learning Systemscited by 2
  26. 2024Impact of ML Optimization Tactics on Greener Pre-Trained ML Modelscited by 2
  27. 2025Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Modelscited by 2
  28. 2025Explainable Artificial Intelligence Techniques for Software Development Lifecycle: A Phase-specific Surveycited by 2
  29. 2025Synthesizing Precise Protocol Specs from Natural Language for Effective Test Generationcited by 2
  30. 2026The Semi-Executable Stack: Agentic Software Engineering and the Expanding Scope of SEcited by 2
  31. 2022Guiding the retraining of convolutional neural networks against adversarial inputscited by 1
  32. 2022Capturing Dependencies within Machine Learning via a Formal Process Modelcited by 1
  33. 2024A Framework to Model ML Engineering Processescited by 1
  34. 2024From Expectation to Habit: Why Do Software Practitioners Adopt Fairness Toolkits?cited by 1
  35. 2025How Do Model Export Formats Impact the Development of ML-Enabled Systems? A Case Study on Model Integrationcited by 1
  36. 2025Aggregating empirical evidence from data strategy studies: a case on model quantizationcited by 1
  37. 2025ReGA: Model-Based Safeguard for LLMs via Representation-Guided Abstractioncited by 1
  38. 2026Computational Compliance for AI Regulation: Blueprint for a New Research Domaincited by 1

50 most cited of 68 All 68 (JSON)