# Human-Centric Research for NLP: Towards a Definition and Guiding Questions

arXiv: https://arxiv.org/abs/2207.04447 · HTML: https://mlsanity.com/paper/2207.04447/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 65 references that are in the map.
- First posted on arXiv 10 Jul 2022.

## Builds on (most cited first, 30 of 32)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation](https://mlsanity.com/paper/2010.01717/) (2020)
- [Explanation-Based Human Debugging of NLP Models: A Survey](https://mlsanity.com/paper/2104.15135/) (2021)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [Neural Open Information Extraction](https://mlsanity.com/paper/1805.04270/) (2018)
- [Putting Humans in the Natural Language Processing Loop: A Survey](https://mlsanity.com/paper/2103.04044/) (2021)
- [Rationale-Augmented Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/1605.04469/) (2016)
- [How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels](https://mlsanity.com/paper/2008.11721/) (2020)
- [Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers](https://mlsanity.com/paper/2010.00667/) (2020)
- [IMoJIE: Iterative Memory-Based Joint Open Information Extraction](https://mlsanity.com/paper/2005.08178/) (2020)
- [OPIEC: An Open Information Extraction Corpus](https://mlsanity.com/paper/1904.12324/) (2019)
- [If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills](https://mlsanity.com/paper/2002.01862/) (2020)
- [SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums](https://mlsanity.com/paper/1906.01727/) (2019)
- [Weakly- and Semi-supervised Evidence Extraction](https://mlsanity.com/paper/2011.01459/) (2020)
- [What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Design](https://mlsanity.com/paper/2007.06718/) (2020)
- [Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models](https://mlsanity.com/paper/2102.00036/) (2021)
- [Decision-Focused Summarization](https://mlsanity.com/paper/2109.06896/) (2021)

## Built on by (most cited first, 9 of 9)

- [Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents](https://mlsanity.com/paper/2402.09205/) (2024)
- [Aligning Language Models with Human Preferences via a Bayesian Approach](https://mlsanity.com/paper/2310.05782/) (2023)
- [Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions](https://mlsanity.com/paper/2502.13780/) (2025)
- [Linking Surface Facts to Large-Scale Knowledge Graphs](https://mlsanity.com/paper/2310.14909/) (2023)
- [Building Better: Avoiding Pitfalls in Developing Language Resources when Data is Scarce](https://mlsanity.com/paper/2410.12691/) (2024)
- [On Synthesizing Data for Context Attribution in Question Answering](https://mlsanity.com/paper/2504.05317/) (2025)
- [Human-artificial intelligence teaming for scientific information extraction from data-driven additive manufacturing research using large language models](https://mlsanity.com/paper/2407.18827/) (2024)
- [Human-Centric NLP or AI-Centric Illusion?: A Critical Investigation](https://mlsanity.com/paper/2412.10939/) (2024)
- [TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action](https://mlsanity.com/paper/2509.15098/) (2025)

## Abstract (arXiv)

With Human-Centric Research (HCR) we can steer research activities so that the research outcome is beneficial for human stakeholders, such as end users. But what exactly makes research human-centric? We address this question by providing a working definition and define how a research pipeline can be split into different stages in which human-centric components can be added. Additionally, we discuss existing NLP with HCR components and define a series of guiding questions, which can serve as starting points for researchers interested in exploring human-centric research approaches. We hope that this work would inspire researchers to refine the proposed definition and to pose other questions that might be meaningful for achieving HCR.