# Putting Humans in the Natural Language Processing Loop: A Survey

arXiv: https://arxiv.org/abs/2103.04044 · HTML: https://mlsanity.com/paper/2103.04044/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [Learning from Dialogue after Deployment: Feed Yourself, Chatbot!](https://mlsanity.com/paper/1901.05415/) (2019)
- [Can Neural Machine Translation be Improved with User Feedback?](https://mlsanity.com/paper/1804.05958/) (2018)
- [Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1804.06512/) (2018)
- [Dialogue Learning With Human-In-The-Loop](https://mlsanity.com/paper/1611.09823/) (2016)
- [Why Didn't You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models](https://mlsanity.com/paper/1905.09864/) (2019)
- [ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems](https://mlsanity.com/paper/1409.4814/) (2014)
- [TopicSifter: Interactive Search Space Reduction Through Targeted Topic Modeling](https://mlsanity.com/paper/1907.12079/) (2019)
- [Counterfactual Learning from Human Proofreading Feedback for Semantic Parsing](https://mlsanity.com/paper/1811.12239/) (2018)
- [Interactive Semantic Featuring for Text Classification](https://mlsanity.com/paper/1606.07545/) (2016)

## Built on by (most cited first, 30 of 40)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus](https://mlsanity.com/paper/2105.02732/) (2021)
- [Explanation-Based Human Debugging of NLP Models: A Survey](https://mlsanity.com/paper/2104.15135/) (2021)
- [Conformal Prediction for Natural Language Processing: A Survey](https://mlsanity.com/paper/2405.01976/) (2024)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Large Language Model-based Human-Agent Collaboration for Complex Task Solving](https://mlsanity.com/paper/2402.12914/) (2024)
- [Interactive Natural Language Processing](https://mlsanity.com/paper/2305.13246/) (2023)
- [Dodrio: Exploring Transformer Models with Interactive Visualization](https://mlsanity.com/paper/2103.14625/) (2021)
- [Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates](https://mlsanity.com/paper/2110.03212/) (2021)
- [Aligning Language Models with Human Preferences via a Bayesian Approach](https://mlsanity.com/paper/2310.05782/) (2023)
- [ProtoTEx: Explaining Model Decisions with Prototype Tensors](https://mlsanity.com/paper/2204.05426/) (2022)
- [Interactive Concept Learning for Uncovering Latent Themes in Large Text Collections](https://mlsanity.com/paper/2305.05094/) (2023)
- [Human-Centric Research for NLP: Towards a Definition and Guiding Questions](https://mlsanity.com/paper/2207.04447/) (2022)
- [Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI](https://mlsanity.com/paper/2308.07213/) (2023)
- [IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models](https://mlsanity.com/paper/2303.03124/) (2023)
- [Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions](https://mlsanity.com/paper/2502.13780/) (2025)
- [KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts](https://mlsanity.com/paper/2403.04758/) (2024)
- [Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks](https://mlsanity.com/paper/2111.01340/) (2021)
- [iScore: Visual Analytics for Interpreting How Language Models Automatically Score Summaries](https://mlsanity.com/paper/2403.04760/) (2024)
- [How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyond](https://mlsanity.com/paper/2501.05714/) (2025)
- [Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agents](https://mlsanity.com/paper/2602.01405/) (2026)
- [On the Security Vulnerabilities of Text-to-SQL Models](https://mlsanity.com/paper/2211.15363/) (2022)
- [Translate your gibberish: black-box adversarial attack on machine translation systems](https://mlsanity.com/paper/2303.10974/) (2023)
- [Facilitating Opinion Diversity through Hybrid NLP Approaches](https://mlsanity.com/paper/2405.09439/) (2024)
- [Uncovering Biases with Reflective Large Language Models](https://mlsanity.com/paper/2408.13464/) (2024)

## Abstract (arXiv)

How can we design Natural Language Processing (NLP) systems that learn from human feedback? There is a growing research body of Human-in-the-loop (HITL) NLP frameworks that continuously integrate human feedback to improve the model itself. HITL NLP research is nascent but multifarious -- solving various NLP problems, collecting diverse feedback from different people, and applying different methods to learn from collected feedback. We present a survey of HITL NLP work from both Machine Learning (ML) and Human-Computer Interaction (HCI) communities that highlights its short yet inspiring history, and thoroughly summarize recent frameworks focusing on their tasks, goals, human interactions, and feedback learning methods. Finally, we discuss future directions for integrating human feedback in the NLP development loop.