mlsanityData to 7 Oct 2026

The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems

arXiv 2105.03354 ↗First posted 7 May 2021cs.AI cs.HCcited by 39

Not on the map. Why?

Abstract (arXiv)

Recent technological advances, especially in the field of machine learning, provide astonishing progress on the road towards artificial general intelligence. However, tasks in current real-world business applications cannot yet be solved by machines alone. We, therefore, identify the need for developing socio-technological ensembles of humans and machines. Such systems possess the ability to accomplish complex goals by combining human and artificial intelligence to collectively achieve superior results and continuously improve by learning from each other. Thus, the need for structured design knowledge for those systems arises. Following a taxonomy development method, this article provides three main contributions: First, we present a structured overview of interdisciplinary research on the role of humans in the machine learning pipeline. Second, we envision hybrid intelligence systems and conceptualize the relevant dimensions for system design for the first time. Finally, we offer useful guidance for system developers during the implementation of such applications.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 16 of its 84 references that are in the map. First posted on arXiv 7 May 2021.

Builds on

16 of 84 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2016Adversarial examples in the physical worldcited by 1,762
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  5. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  6. 2016The Mythos of Model Interpretabilitycited by 716
  7. 2017Large-Scale Evolution of Image Classifierscited by 348
  8. 2015The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classificationcited by 76
  9. 2017Machine Teaching: A New Paradigm for Building Machine Learning Systemscited by 41
  10. 2017Towards Practical Verification of Machine Learning: The Case of Computer Vision Systemscited by 19
  11. 2016On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systemscited by 18
  12. 2016Debugging Machine Learning Taskscited by 9
4 more
  1. 2015Crowdsourcing Feature Discovery via Adaptively Chosen Comparisonscited by 5
  2. 2014An Active Learning Approach for Jointly Estimating Worker Performance and Annotation Reliability with Crowdsourced Datacited by 3
  3. 2017Interpretable and Pedagogical Examplescited by 2
  4. 2016Co-active Learning to Adapt Humanoid Movement for Manipulationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 6; 2024: 7; 2025: 10; 2026: 9.

39 papers.

Sort:

By area:

  1. 2021Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AIcited by 38
  2. 2024Evaluating Human-AI Collaboration: A Review and Methodological Frameworkcited by 25
  3. 2023Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experiencecited by 18
  4. 2024AI-enhanced Collective Intelligencecited by 10
  5. 2022A Test for Evaluating Performance in Human-Computer Systemscited by 8
  6. 2023Conversational AI-Powered Design: ChatGPT as Designer, User, and Productcited by 8
  7. 2024Unpacking Human-AI interactions: From interaction primitives to a design spacecited by 7
  8. 2024Interrogating AI: Characterizing Emergent Playful Interactions with ChatGPTcited by 5
  9. 2025"What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasetscited by 5
  10. 2022Painting the black box white: experimental findings from applying XAI to an ECG reading settingcited by 4
  11. 2023Monotonicity for AI ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and financecited by 4
  12. 2023IMAD: IMage-Augmented multi-modal Dialoguecited by 4
27 more
  1. 2024On the Ethical Considerations of Generative Agentscited by 4
  2. 2022Artificial intelligence in government: Concepts, standards, and a unified frameworkcited by 3
  3. 2025Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomycited by 3
  4. 2025The Epistemological Consequences of Large Language Models: Rethinking collective intelligence and institutional knowledgecited by 2
  5. 2022Modeling Human Behavior Part I -- Learning and Belief Approachescited by 1
  6. 2022Rethinking and Recomputing the Value of Machine Learning Modelscited by 1
  7. 2022Trustworthy Human Computation: A Surveycited by 1
  8. 2023Improving the Efficiency of Human-in-the-Loop Systems: Adding Artificial to Human Expertscited by 1
  9. 2024The two-way knowledge interaction interface between humans and neural networkscited by 1
  10. 2023Eliciting Model Steering Interactions from Users via Data and Visual Design Probescited by 0
  11. 2024Human-Generative AI Collaborative Problem Solving Who Leads and How Students Perceive the Interactionscited by 0
  12. 2025A Beautiful Mind: Principles and Strategies for AI-Augmented Human Reasoningcited by 0
  13. 2025A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversationscited by 0
  14. 2025Human aversion? Do AI Agents Judge Identity More Harshly Than Performancecited by 0
  15. 2025Human-Centered Human-AI Collaboration (HCHAC)cited by 0
  16. 2025Exploring Human-AI Collaboration Using Mental Models of Early Adopters of Multi-Agent Generative AI Toolscited by 0
  17. 2025The Rise of the Knowledge Sculptor: A New Archetype for Knowledge Work in the Age of Generative AIcited by 0
  18. 2025Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generationcited by 0
  19. 2026From Transcripts to AI Agents: Knowledge Extraction, RAG Integration, and Robust Evaluation of Conversational AI Assistantscited by 0
  20. 2026Ghost Framing Theory: Exploring the role of generative AI in new venture rhetorical legitimationcited by 0
  21. 2026LLM-Powered Workflow Optimization for Multidisciplinary Software Development: An Automotive Industry Case Studycited by 0
  22. 2026Human-AI Collaboration for Estimating Scientific Replicabilitycited by 0
  23. 2026AI in the Workplace: The Impact of AI on Perceived Job Decency and Meaningfulnesscited by 0
  24. 2026The New Social Image: How AI Competency and AI Proactivity Influence Self- and Peer-Perceptions in the Workplacecited by 0
  25. 2026Comparing Socio-technical Design Principles with Guidelines for Human-centered AIcited by 0
  26. 2026Artificial Intelligence and Innovation Ecosystem: Evolutionary Developments, Challenges, and Future Directionscited by 0
  27. 2026Graph-Based Inference for Feedback-Driven Word Deduction: A Scalable Framework for the Jotto Problemcited by 0