# How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact

arXiv: https://arxiv.org/abs/2106.02359 · HTML: https://mlsanity.com/paper/2106.02359/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 96 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://mlsanity.com/paper/2004.02984/) (2020)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [De-identification of Patient Notes with Recurrent Neural Networks](https://mlsanity.com/paper/1606.03475/) (2016)
- [Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks](https://mlsanity.com/paper/1801.07772/) (2018)
- [Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms](https://mlsanity.com/paper/2007.07996/) (2020)
- [NeuroNER: an easy-to-use program for named-entity recognition based on neural networks](https://mlsanity.com/paper/1705.05487/) (2017)
- [Learning to Prescribe Interventions for Tuberculosis Patients Using Digital Adherence Data](https://mlsanity.com/paper/1902.01506/) (2019)
- [Predicting Economic Development using Geolocated Wikipedia Articles](https://mlsanity.com/paper/1905.01627/) (2019)
- [Expressive Interviewing: A Conversational System for Coping with COVID-19](https://mlsanity.com/paper/2007.03819/) (2020)
- [HappyDB: A Corpus of 100,000 Crowdsourced Happy Moments](https://mlsanity.com/paper/1801.07746/) (2018)
- [Linking Artificial Intelligence Principles](https://mlsanity.com/paper/1812.04814/) (2018)
- [Tracking COVID-19 using online search](https://mlsanity.com/paper/2003.08086/) (2020)
- [Quantifying the Effects of COVID-19 on Mental Health Support Forums](https://mlsanity.com/paper/2009.04008/) (2020)
- [AWS CORD-19 Search: A Neural Search Engine for COVID-19 Literature](https://mlsanity.com/paper/2007.09186/) (2020)
- [Learning Twitter User Sentiments on Climate Change with Limited Labeled Data](https://mlsanity.com/paper/1904.07342/) (2019)
- [Happiness Entailment: Automating Suggestions for Well-Being](https://mlsanity.com/paper/1907.10036/) (2019)
- [Fast, Structured Clinical Documentation via Contextual Autocomplete](https://mlsanity.com/paper/2007.15153/) (2020)

## Built on by (most cited first, 18 of 18)

- [Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity](https://mlsanity.com/paper/2301.12867/) (2023)
- [The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail](https://mlsanity.com/paper/2110.08300/) (2021)
- [Quantifying the Dialect Gap and its Correlates Across Languages](https://mlsanity.com/paper/2310.15135/) (2023)
- [Logical Fallacy Detection](https://mlsanity.com/paper/2202.13758/) (2022)
- [Towards Climate Awareness in NLP Research](https://mlsanity.com/paper/2205.05071/) (2022)
- [The Call for Socially Aware Language Technologies](https://mlsanity.com/paper/2405.02411/) (2024)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good](https://mlsanity.com/paper/2305.05471/) (2023)
- [Designing Language Technologies for Social Good: The Road not Taken](https://mlsanity.com/paper/2110.07444/) (2021)
- [Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational Impact in LLM Tutors](https://mlsanity.com/paper/2606.16206/) (2026)
- [FinTech for Social Good: A Research Agenda from NLP Perspective](https://mlsanity.com/paper/2211.06431/) (2022)
- [Natural Language Processing for Policymaking](https://mlsanity.com/paper/2302.03490/) (2023)
- [Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP Artefacts](https://mlsanity.com/paper/2505.18677/) (2025)
- [HatePRISM: Policies, Platforms, and Research Integration. Advancing NLP for Hate Speech Proactive Mitigation](https://mlsanity.com/paper/2507.04350/) (2025)
- [Good Intentions Beyond ACL: Who Does NLP for Social Good, and Where?](https://mlsanity.com/paper/2510.04434/) (2025)
- [Demarked: A Strategy for Enhanced Abusive Speech Moderation through Counterspeech, Detoxification, and Message Management](https://mlsanity.com/paper/2406.19543/) (2024)
- [What is the social benefit of hate speech detection research? A Systematic Review](https://mlsanity.com/paper/2409.17467/) (2024)
- [Operationalizing AI for Good: Spotlight on Deployment and Integration of AI Models in Humanitarian Work](https://mlsanity.com/paper/2507.15823/) (2025)

## Abstract (arXiv)

Recent years have seen many breakthroughs in natural language processing (NLP), transitioning it from a mostly theoretical field to one with many real-world applications. Noting the rising number of applications of other machine learning and AI techniques with pervasive societal impact, we anticipate the rising importance of developing NLP technologies for social good. Inspired by theories in moral philosophy and global priorities research, we aim to promote a guideline for social good in the context of NLP. We lay the foundations via the moral philosophy definition of social good, propose a framework to evaluate the direct and indirect real-world impact of NLP tasks, and adopt the methodology of global priorities research to identify priority causes for NLP research. Finally, we use our theoretical framework to provide some practical guidelines for future NLP research for social good. Our data and code are available at http://github.com/zhijing-jin/nlp4sg_acl2021. In addition, we curate a list of papers and resources on NLP for social good at https://github.com/zhijing-jin/NLP4SocialGood_Papers.