mlsanityData to 14 Oct 2026

An Inclusive Notion of Text

arXiv 2211.05604 ↗First posted 10 Nov 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Natural language processing (NLP) researchers develop models of grammar, meaning and communication based on written text. Due to task and data differences, what is considered text can vary substantially across studies. A conceptual framework for systematically capturing these differences is lacking. We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. Towards that goal, we propose common terminology to discuss the production and transformation of textual data, and introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling. We apply this taxonomy to survey existing work that extends the notion of text beyond the conservative language-centered view. We outline key desiderata and challenges of the emerging inclusive approach to text in NLP, and suggest community-level reporting as a crucial next step to consolidate the discussion.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 59 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

38 of 59 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2018Datasheets for Datasetscited by 1,386
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  8. 2018Model Cards for Model Reportingcited by 1,080
  9. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  10. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  11. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  12. 2021Datasets: A Community Library for Natural Language Processingcited by 530
26 more
  1. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  2. 2022Multimodal Learning with Transformers: A Surveycited by 400
  3. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  4. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 136
  5. 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 130
  6. 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 118
  7. 2022PEER: A Collaborative Language Modelcited by 86
  8. 2021HTLM: Hyper-Text Pre-Training and Prompting of Language Modelscited by 58
  9. 2021Just What do You Think You're Doing, Dave?' A Checklist for Responsible Data Use in NLPcited by 47
  10. 2021MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understandingcited by 45
  11. 2019Argument Mining for Understanding Peer Reviewscited by 41
  12. 2021FRUIT: Faithfully Reflecting Updated Information in Textcited by 39
  13. 2022Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Reviewcited by 37
  14. 2021DISAPERE: A Dataset for Discourse Structure in Peer Review Discussionscited by 36
  15. 2016What to do about non-standard (or non-canonical) language in NLPcited by 35
  16. 2019Does My Rebuttal Matter? Insights from a Major NLP Conferencecited by 31
  17. 2021MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label settingcited by 30
  18. 2021VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groupscited by 27
  19. 2018WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discoursecited by 26
  20. 2022HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Informationcited by 26
  21. 2022NewsEdits: A News Article Revision Dataset and a Document-Level Reasoning Challengecited by 20
  22. 2020Generalizing Cross-Document Event Coreference Resolution Across Multiple Corporacited by 18
  23. 2022ChapterBreak: A Challenge Dataset for Long-Range Language Modelscited by 18
  24. 2022Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyondcited by 12
  25. 2020NLP Scholar: An Interactive Visual Explorer for Natural Language Processing Literaturecited by 8
  26. 2020SemEval-2020 Task 10: Emphasis Selection for Written Text in Visual Mediacited by 3

Built on by

Not cited yet.