mlsanityData to 7 Oct 2026

A Survey of Human-in-the-loop for Machine Learning

arXiv 2108.00941 ↗First posted 2 Aug 2021cs.LGcited by 214

Not on the map. Why?

Cited by 20 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Human-in-the-loop aims to train an accurate prediction model with minimum cost by integrating human knowledge and experience. Humans can provide training data for machine learning applications and directly accomplish tasks that are hard for computers in the pipeline with the help of machine-based approaches. In this paper, we survey existing works on human-in-the-loop from a data perspective and classify them into three categories with a progressive relationship: (1) the work of improving model performance from data processing, (2) the work of improving model performance through interventional model training, and (3) the design of the system independent human-in-the-loop. Using the above categorization, we summarize major approaches in the field; along with their technical strengths/ weaknesses, we have simple classification and discussion in natural language processing, computer vision, and others. Besides, we provide some open challenges and opportunities. This survey intends to provide a high-level summarization for human-in-the-loop and motivates interested readers to consider approaches for designing effective human-in-the-loop solutions.

Key facts. Cited by 214 papers in the map (data through 7 Oct 2026). Builds on 84 of its 242 references that are in the map. First posted on arXiv 2 Aug 2021.

Builds on

84 of 242 references.

References · 84

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2015Fast R-CNNcited by 2,930
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  11. 2018World Modelscited by 1,444
  12. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
38 more
  1. 2017Deep Image Priorcited by 1,097
  2. 2021Transformers in Vision: A Surveycited by 1,009
  3. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  4. 2020Efficient Transformers: A Surveycited by 784
  5. 2021Domain Generalization: A Surveycited by 760
  6. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  7. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  8. 2019Object Detection in 20 Years: A Surveycited by 528
  9. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  10. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  11. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  12. 2018A Survey on Deep Transfer Learningcited by 417
  13. 2017NIMA: Neural Image Assessmentcited by 387
  14. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  15. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  16. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  17. 2016One-Shot Video Object Segmentationcited by 233
  18. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 169
  19. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
  20. 2019EdgeConnect: Generative Image Inpainting with Adversarial Edge Learningcited by 161
  21. 2016Deep Interactive Object Selectioncited by 160
  22. 2016Learning Video Object Segmentation from Static Imagescited by 151
  23. 2019A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysiscited by 150
  24. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  25. 2017Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuningcited by 110
  26. 2019Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesiscited by 108
  27. 2019Learning from Dialogue after Deployment: Feed Yourself, Chatbot!cited by 102
  28. 2018Shift-Net: Image Inpainting via Deep Feature Rearrangementcited by 87
  29. 2021Improving Transferability of Adversarial Patches on Face Recognition with Generative Modelscited by 80
  30. 2017Learning Video Object Segmentation with Visual Memorycited by 66
  31. 2019Multimodal Self-Supervised Learning for Medical Image Analysiscited by 57
  32. 2020Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learningcited by 53
  33. 2018Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failurecited by 49
  34. 2019Attentional Encoder Network for Targeted Sentiment Classificationcited by 48
  35. 2018The 2018 DAVIS Challenge on Video Object Segmentationcited by 47
  36. 2016Dialogue Learning With Human-In-The-Loopcited by 46
  37. 2020Deep Hough-Transform Line Priorscited by 44
  38. 2019Leveraging Human Guidance for Deep Reinforcement Learning Taskscited by 42

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 24; 2023: 48; 2024: 52; 2025: 47; 2026: 42.

214 papers.

Sort:

By area:

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  2. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  3. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  4. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  5. 2025M+: Extending MemoryLLM with Scalable Long-Term Memorycited by 51
  6. 2025FLEX: Continuous Agent Evolution via Forward Learning from Experiencecited by 51
  7. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  8. 2023LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzlescited by 41
  9. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  10. 2023Natural Language Processing in the Legal Domaincited by 38
  11. 2024Large Language Model-based Human-Agent Collaboration for Complex Task Solvingcited by 38
  12. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
38 more
  1. 2024Conformal Prediction Sets Improve Human Decision Makingcited by 35
  2. 2023Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomiescited by 34
  3. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 34
  4. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  5. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 28
  6. 2022A Rationale-Centric Framework for Human-in-the-loop Machine Learningcited by 27
  7. 2023Interactive Explanations by Conflict Resolution via Argumentative Exchangescited by 24
  8. 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 24
  9. 2023Towards Theoretical Understanding of Inverse Reinforcement Learningcited by 21
  10. 2023Reinforcement Learning with Human Feedback for Realistic Traffic Simulationcited by 20
  11. 2023Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Textcited by 19
  12. 2023Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake?cited by 18
  13. 2024Evaluating Large Language Models as Virtual Annotators for Time-series Physical Sensing Datacited by 17
  14. 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoningcited by 16
  15. 2024Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Makingcited by 14
  16. 2024Position: A Call to Action for a Human-Centered AutoML Paradigmcited by 14
  17. 2022Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedbackcited by 13
  18. 2023AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challengescited by 13
  19. 2023Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Taskscited by 13
  20. 2025LLM-based Automated Grading with Human-in-the-Loopcited by 13
  21. 2023Static Code Analysis in the AI Era: An In-depth Exploration of the Concept, Function, and Potential of Intelligent Code Analysis Agentscited by 12
  22. 2024From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?cited by 11
  23. 2024Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLPcited by 11
  24. 2023Learning a Universal Human Prior for Dexterous Manipulation from Human Preferencecited by 10
  25. 2024TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detectioncited by 10
  26. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  27. 2022Human-Centric Research for NLP: Towards a Definition and Guiding Questionscited by 9
  28. 2023ChatGPT is on the Horizon: Could a Large Language Model be Suitable for Intelligent Traffic Safety Research and Applications?cited by 9
  29. 2025AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Toolscited by 8
  30. 2023DMOps: Data Management Operation and Recipescited by 7
  31. 2024Unpacking Human-AI interactions: From interaction primitives to a design spacecited by 7
  32. 2024Learning to Complement and to Defer to Multiple Userscited by 7
  33. 2024AI-generated Image Quality Assessment in Visual Communicationcited by 7
  34. 2025SCALE: Towards Collaborative Content Analysis in Social Science with Large Language Model Agents and Human Interventioncited by 7
  35. 2022Causal Discovery and Knowledge Injection for Contestable Neural Networks (with Appendices)cited by 6
  36. 2022Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Controlcited by 6
  37. 2024ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotationcited by 6
  38. 2024Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunitiescited by 6

50 most cited of 214 All 214 (JSON)