mlsanityData to 7 Oct 2026

Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities

arXiv 1804.05892 ↗First posted 16 Apr 2018cs.DBcited by 20

Not on the map. Why?

Abstract (arXiv)

Development of machine learning (ML) workflows is a tedious process of iterative experimentation: developers repeatedly make changes to workflows until the desired accuracy is attained. We describe our vision for a "human-in-the-loop" ML system that accelerates this process: by intelligently tracking changes and intermediate results over time, such a system can enable rapid iteration, quick responsive feedback, introspection and debugging, and background execution and automation. We finally describe Helix, our preliminary attempt at such a system that has already led to speedups of up to 10x on typical iterative workflows against competing systems.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 5; 2024: 3; 2025: 3; 2026: 2.

20 papers.

Sort:

By area:

  1. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  2. 2022High-Definition Map Generation Technologies For Autonomous Drivingcited by 14
  3. 2023Evaluation of Induced Expert Knowledge in Causal Structure Learning by NOTEARScited by 8
  4. 2022Computing Multiple Image Reconstructions with a Single Hypernetworkcited by 7
  5. 2024Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Brakingcited by 5
  6. 2024Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approachcited by 3
  7. 2025Stochastic Trajectory Prediction under Unstructured Constraintscited by 3
  8. 2025VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DRcited by 3
  9. 2021Active learning for medical code assignmentcited by 2
  10. 2024Smart Language Agents in Real-World Planningcited by 1
  11. 2025HP-BERT: A framework for longitudinal study of Hinduphobia on social media via language modelscited by 1
  12. 2026Interactive Analysis of Global Explanations using Aggregated Class Activation Maps for Network Datacited by 1
8 more
  1. 2021Towards Model-informed Precision Dosing with Expert-in-the-loop Machine Learningcited by 0
  2. 2021Power of human-algorithm collaboration in solving combinatorial optimization problemscited by 0
  3. 2022A Blockchain Protocol for Human-in-the-Loop AIcited by 0
  4. 2023Transforming Unstructured Text into Data with Context Rule Assisted Machine Learning (CRAML)cited by 0
  5. 2023MaskSearch: Querying Image Masks at Scalecited by 0
  6. 2023Efficient Failure Pattern Identification of Predictive Algorithmscited by 0
  7. 2023Assurance for Deployed Continual Learning Systemscited by 0
  8. 2026The Machine Learning Canvas: Empirical Findings on Why Strategy Matters More Than AI Code Generationcited by 0