mlsanityData to 7 Oct 2026

POTATO: The Portable Text Annotation Tool

arXiv 2212.08620 ↗First posted 16 Dec 2022cs.CL cs.AI cs.CYcited by 64

Not on the map. Why?

Abstract (arXiv)

We present POTATO, the Portable text annotation tool, a free, fully open-sourced annotation system that 1) supports labeling many types of text and multimodal data; 2) offers easy-to-configure features to maximize the productivity of both deployers and annotators (convenient templates for common ML/NLP tasks, active learning, keypress shortcuts, keyword highlights, tooltips); and 3) supports a high degree of customization (editable UI, inserting pre-screening questions, attention and qualification tests). Experiments over two annotation tasks suggest that POTATO improves labeling speed through its specially-designed productivity features, especially for long documents and complex tasks. POTATO is available at https://github.com/davidjurgens/potato and will continue to be updated.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 18 of its 45 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

18 of 45 references.

References · 18

  1. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  2. 2018Model Cards for Model Reportingcited by 1,068
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  4. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  6. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  7. 2019The Ethics of AI Ethics -- An Evaluation of Guidelinescited by 129
  8. 2021Beyond Fair Pay: Ethical Implications of NLP Crowdsourcingcited by 52
  9. 2019Roles for Computing in Social Changecited by 49
  10. 2020Masakhane -- Machine Translation For Africacited by 48
  11. 2021Modeling Framing in Immigration Discourse on Social Mediacited by 37
  12. 2021Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communicationscited by 27
6 more
  1. 2020Quantifying Intimacy in Languagecited by 25
  2. 2021Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversationscited by 20
  3. 2021LightTag: Text Annotation Platformcited by 16
  4. 2022SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysiscited by 14
  5. 2022Modeling Information Change in Science Communication with Semantically Matched Paraphrasescited by 14
  6. 2021An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialogcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 18; 2025: 26; 2026: 6.

64 papers.

Sort:

By area:

  1. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  2. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  3. 2023Human Feedback is not Gold Standardcited by 82
  4. 2023An Emulator for Fine-Tuning Large Language Models using Small Language Modelscited by 61
  5. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  6. 2023When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Datasetcited by 51
  7. 2025Modifying Large Language Model Post-Training for Diverse Creative Writingcited by 39
  8. 2023Chinese Open Instruction Generalist: A Preliminary Releasecited by 35
  9. 2024AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queriescited by 32
  10. 2024WildFeedback: Aligning LLMs With In-situ User Interactions And Feedbackcited by 31
  11. 2025BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languagescited by 27
  12. 2024SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakescited by 24
38 more
  1. 2024MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflowscited by 19
  2. 2024Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understandingcited by 17
  3. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  4. 2025Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoralcited by 14
  5. 2025CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understandingcited by 14
  6. 2025ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environmentscited by 13
  7. 2024Fearful Falcons and Angry Llamas: Emotion Category Annotations of Arguments by Humans and LLMscited by 11
  8. 2026Pearmut: Human Evaluation of Translation Made Trivialcited by 11
  9. 2024A Reality Check on Context Utilisation for Retrieval-Augmented Generationcited by 9
  10. 2024Understanding Fine-grained Distortions in Reports of Scientific Findingscited by 8
  11. 2023DMOps: Data Management Operation and Recipescited by 7
  12. 2023The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Changecited by 7
  13. 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Datasetcited by 7
  14. 2023Your spouse needs professional help: Determining the Contextual Appropriateness of Messages through Modeling Social Relationshipscited by 6
  15. 2023Thresh: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluationcited by 6
  16. 2024Mapping the Podcast Ecosystem with the Structured Podcast Research Corpuscited by 6
  17. 2025Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbolscited by 6
  18. 2025Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMscited by 6
  19. 2024Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questionscited by 5
  20. 2025What is Stigma Attributed to? A Theory-Grounded, Expert-Annotated Interview Corpus for Demystifying Mental-Health Stigmacited by 5
  21. 2023Can GPT models Follow Human Summarization Guidelines? A Study for Targeted Communication Goalscited by 4
  22. 2023StorySparkQA: Expert-Annotated QA Pairs with Real-World Knowledge for Children's Story-Based Learningcited by 4
  23. 2024An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessmentcited by 4
  24. 2025iNews: A Multimodal Dataset for Modeling Personalized Affective Responses to Newscited by 4
  25. 2025POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarizationcited by 4
  26. 2024Cross-cultural Inspiration Detection and Analysis in Real and LLM-generated Social Media Datacited by 3
  27. 2025Social Story Frames: Contextual Reasoning about Narrative Intent and Receptioncited by 3
  28. 2024Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questionscited by 2
  29. 2024The Language of Trauma: Modeling Traumatic Event Descriptions Across Domains with Explainable AIcited by 2
  30. 2025Reassessing Active Learning Adoption in Contemporary NLP: A Community Surveycited by 2
  31. 2025QUARTZ : QA-based Unsupervised Abstractive Refinement for Task-oriented Dialogue Summarizationcited by 2
  32. 2025The Collective Turing Test: Large Language Models Can Generate Realistic Multi-User Discussionscited by 2
  33. 2025ORCA: Open-ended Response Correctness Assessment for Audio Question Answeringcited by 2
  34. 2024EEVEE: An Easy Annotation Tool for Natural Language Processingcited by 1
  35. 2024BotEval: Facilitating Interactive Human Evaluationcited by 1
  36. 2024MetaphorShare: A Dynamic Collaborative Repository of Open Metaphor Datasetscited by 1
  37. 2025Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related taskscited by 1
  38. 2025Enhancing Multi-Label Emotion Analysis and Corresponding Intensities for Ethiopian Languagescited by 1

50 most cited of 64 All 64 (JSON)