A Major Obstacle for NLP Research: Let's Talk about Time Allocation!
Not on the map. Why?
Abstract (arXiv)
The field of natural language processing (NLP) has grown over the last few years: conferences have become larger, we have published an incredible amount of papers, and state-of-the-art research has been implemented in a large variety of customer-facing products. However, this paper argues that we have been less successful than we should have been and reflects on where and how the field fails to tap its full potential. Specifically, we demonstrate that, in recent years, subpar time allocation has been a major obstacle for NLP research. We outline multiple concrete problems together with their negative consequences and, importantly, suggest remedies to improve the status quo. We hope that this paper will be a starting point for discussions around which common practices are -- or are not -- beneficial for NLP research.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 30 Nov 2022.
Builds on
40 of 66 references.
References · 40
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2020Experience Grounds Languagecited by 242
- 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
- 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 203
- 2020DART: Open-Domain Structured Data Record to Text Generationcited by 145
- 2018Troubling Trends in Machine Learning Scholarshipcited by 108
- 2022The Fallacy of AI Functionalitycited by 88
- 2020Continual Learning in Task-Oriented Dialogue Systemscited by 83
- 2020ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corporacited by 63
- 2021MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Taskscited by 62
28 more
- 2021Changing the World by Changing the Datacited by 51
- 2021Robust Open-Vocabulary Translation from Visual Text Representationscited by 47
- 2021Aspect-Controllable Opinion Summarizationcited by 47
- 2021Finding a Balanced Degree of Automation for Summary Evaluationcited by 43
- 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
- 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36
- 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
- 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35
- 2021Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policycited by 32
- 2020CR-Walker: Tree-Structured Graph Reasoning and Dialog Acts for Conversational Recommendationcited by 31
- 2021ConvFiT: Conversational Fine-Tuning of Pretrained Language Modelscited by 25
- 2021Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoderscited by 24
- 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
- 2021How Much Coffee Was Consumed During EMNLP 2019? Fermi Problems: A New Reasoning Challenge for AIcited by 22
- 2021End-to-End Learning of Flowchart Grounded Task-Oriented Dialogscited by 20
- 2021Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answeringcited by 19
- 2021Investigating Robustness of Dialog Models to Popular Figurative Language Constructscited by 19
- 2021Refocusing on Relevance: Personalization in NLGcited by 18
- 2021Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLUcited by 17
- 2021Cryptonite: A Cryptic Crossword Benchmark for Extreme Ambiguity in Languagecited by 15
- 2021Towards Making the Most of Dialogue Characteristics for Neural Chat Translationcited by 12
- 2021Cross-lingual Intermediate Fine-tuning improves Dialogue State Trackingcited by 11
- 2021Detecting Speaker Personas from Conversational Textscited by 8
- 2021It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Datacited by 7
- 2021We've had this conversation before: A Novel Approach to Measuring Dialog Similaritycited by 7
- 2021Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Textcited by 6
- 2021Coloring the Black Box: What Synesthesia Tells Us about Character Embeddingscited by 1
- 2021"Is My Mic On?" Preparing SE Students for Collaborative Remote Work and Hybrid Team Communicationcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.