mlsanityData to 7 Oct 2026

How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directions

arXiv 2409.07192 ↗First posted 11 Sep 2024cs.SE cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence (AI) permeates all fields of life, which resulted in new challenges in requirements engineering for artificial intelligence (RE4AI), e.g., the difficulty in specifying and validating requirements for AI or considering new quality requirements due to emerging ethical implications. It is currently unclear if existing RE methods are sufficient or if new ones are needed to address these challenges. Therefore, our goal is to provide a comprehensive overview of RE4AI to researchers and practitioners. What has been achieved so far, i.e., what practices are available, and what research gaps and challenges still need to be addressed? To achieve this, we conducted a systematic mapping study combining query string search and extensive snowballing. The extracted data was aggregated, and results were synthesized using thematic analysis. Our selection process led to the inclusion of 126 primary studies. Existing RE4AI research focuses mainly on requirements analysis and elicitation, with most practices applied in these areas. Furthermore, we identified requirements specification, explainability, and the gap between machine learning engineers and end-users as the most prevalent challenges, along with a few others. Additionally, we proposed seven potential research directions to address these challenges. Practitioners can use our results to identify and select suitable RE methods for working on their AI-based systems, while researchers can build on the identified gaps and research directions to push the field forward.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 33 of its 180 references that are in the map. First posted on arXiv 11 Sep 2024.

Builds on

33 of 180 references.

References · 33

  1. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 251
  2. 2021Software Engineering for AI-Based Systems: A Surveycited by 68
  3. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  4. 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
  5. 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
  6. 2019Requirements Engineering for Machine Learning: Perspectives from Data Scientistscited by 49
  7. 2021Perfection Not Required? Human-AI Partnerships in Code Translationcited by 42
  8. 2020Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systemscited by 36
  9. 2019Requirements Engineering Challenges in Building AI-Based Complex Systemscited by 21
  10. 2020Towards Guidelines for Assessing Qualities of Machine Learning Systemscited by 16
  11. 2020Software engineering for artificial intelligence and machine learning software: A systematic literature reviewcited by 15
  12. 2020Explanation Ontology: A Model of Explanations for User-Centered AIcited by 13
21 more
  1. 2019Towards a Framework to Manage Perceptual Uncertainty for Safe Automated Drivingcited by 12
  2. 2022If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Componentscited by 11
  3. 2019Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systemscited by 7
  4. 2022Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliancecited by 7
  5. 2023Tailoring Requirements Engineering for Responsible AIcited by 7
  6. 2023Requirements Engineering Framework for Human-centered Artificial Intelligence Software Systemscited by 7
  7. 2021Requirement Engineering Challenges for AI-intense Systems Developmentcited by 6
  8. 2021Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Reviewcited by 4
  9. 2021Towards Artefact-based Requirements Engineering for Data-Centric Systemscited by 4
  10. 2021Identifying Roles, Requirements and Responsibilities in Trustworthy AI Systemscited by 4
  11. 2022Can Requirements Engineering Support Explainable Artificial Intelligence? Towards a User-Centric Approach for Explainability Requirementscited by 4
  12. 2023An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applicationscited by 4
  13. 2021Towards Risk Modeling for Collaborative AIcited by 3
  14. 2021Pairing Conceptual Modeling with Machine Learningcited by 3
  15. 2022Non-Functional Requirements for Machine Learning: An Exploration of System Scope and Interestcited by 3
  16. 2022RESAM: Requirements Elicitation and Specification for Deep-Learning Anomaly Models with Applications to UAV Flight Controllerscited by 3
  17. 2021Designer-User Communication for XAI: An epistemological approach to discuss XAI designcited by 2
  18. 2022Towards Perspective-Based Specification of Machine Learning-Enabled Systemscited by 2
  19. 2023Requirements Engineering for Automotive Perception Systems: an Interview Studycited by 1
  20. 2023Towards Audit Requirements for AI-based Systems in Mobility Applicationscited by 1
  21. 2023Defining Quality Requirements for a Trustworthy AI Wildflower Monitoring Platformcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.