mlsanityData to 7 Oct 2026

DocBERT: BERT for Document Classification

arXiv 1904.08398 ↗First posted 17 Apr 2019cs.CLcited by 68

Not on the map. Why?

Abstract (arXiv)

We present, to our knowledge, the first application of BERT to document classification. A few characteristics of the task might lead one to think that BERT is not the most appropriate model: syntactic structures matter less for content categories, documents can often be longer than typical BERT input, and documents often have multiple labels. Nevertheless, we show that a straightforward classification model using BERT is able to achieve the state of the art across four popular datasets. To address the computational expense associated with BERT inference, we distill knowledge from BERT-large to small bidirectional LSTMs, reaching BERT-base parity on multiple datasets using 30x fewer parameters. The primary contribution of our paper is improved baselines that can provide the foundation for future work.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 11; 2023: 12; 2024: 5; 2025: 2; 2026: 2.

68 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  2. 2022Revisiting Transformer-based Models for Long Document Classificationcited by 48
  3. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  4. 2021Scaling Federated Learning for Fine-tuning of Large Language Modelscited by 36
  5. 2023DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patientscited by 29
  6. 2021Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNetcited by 27
  7. 2021Automated scholarly paper review: Concepts, technologies, and challengescited by 21
  8. 2021FakeFlow: Fake News Detection by Modeling the Flow of Affective Informationcited by 20
  9. 2021Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learningcited by 15
  10. 2021Incorporating Uncertainty in Learning to Defer Algorithms for Safe Computer-Aided Diagnosiscited by 15
  11. 2021"Short is the Road that Leads from Fear to Hate": Fear Speech in Indian WhatsApp Groupscited by 13
  12. 2021Automated Quality Assessment of Cognitive Behavioral Therapy Sessions Through Highly Contextualized Language Representationscited by 13
38 more
  1. 2021Comparative Study of Long Document Classificationcited by 12
  2. 2021Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Reviewcited by 11
  3. 2021You too Brutus! Trapping Hateful Users in Social Media: Challenges, Solutions & Insightscited by 11
  4. 2022Improving Short Text Classification With Augmented Data Using GPT-3cited by 11
  5. 2021Comparison of Czech Transformers on Text Classification Taskscited by 10
  6. 2023FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learningcited by 10
  7. 2022Automated Identification of Toxic Code Reviews Using ToxiCRcited by 8
  8. 2021Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviewscited by 7
  9. 2021Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Researchcited by 7
  10. 2021Unsupervised Matching of Data and Textcited by 7
  11. 2022Hierarchical Neural Network Approaches for Long Document Classificationcited by 7
  12. 2022Less is More: Parameter-Free Text Classification with Gzipcited by 7
  13. 2021Three-level Hierarchical Transformer Networks for Long-sequence and Multiple Clinical Documents Classificationcited by 6
  14. 2021PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptorscited by 6
  15. 2021Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentationcited by 6
  16. 2022Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Modelscited by 6
  17. 2022Protoformer: Embedding Prototypes for Transformerscited by 6
  18. 2023ATLAS: Automatically Detecting Discrepancies Between Privacy Policies and Privacy Labelscited by 5
  19. 2023Well-calibrated Confidence Measures for Multi-label Text Classification with a Large Number of Labelscited by 5
  20. 2021Lacking the embedding of a word? Look it up into a traditional dictionarycited by 4
  21. 2024Machine Unlearning for Document Classificationcited by 4
  22. 2021Classifying Long Clinical Documents with Pre-trained Transformerscited by 3
  23. 2021Sentiment analysis in tweets: an assessment study from classical to modern text representation modelscited by 3
  24. 2021Efficient Document Image Classification Using Region-Based Graph Neural Networkcited by 3
  25. 2021Large-Scale News Classification using BERT Language Model: Spark NLP Approachcited by 3
  26. 2022Can Language Representation Models Think in Bets?cited by 3
  27. 2023Ticket-BERT: Labeling Incident Management Tickets with Language Modelscited by 3
  28. 2024PecSched: Preemptive and Efficient Cluster Scheduling for LLM Inferencecited by 3
  29. 2025Unsupervised Document and Template Clustering using Multimodal Embeddingscited by 3
  30. 2021On-Device Document Classification using multimodal featurescited by 2
  31. 2021One System to Rule them All: a Universal Intent Recognition System for Customer Service Chatbotscited by 2
  32. 2022Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probescited by 2
  33. 2023Multi label classification of Artificial Intelligence related patents using Modified D2SBERT and Sentence Attention mechanismcited by 2
  34. 2024TacoERE: Cluster-aware Compression for Event Relation Extractioncited by 2
  35. 2024FDC: Fast KV Dimensionality Compression for Efficient LLM Inferencecited by 2
  36. 2025Label-semantics Aware Generative Approach for Domain-Agnostic Multilabel Classificationcited by 2
  37. 2021Does a Hybrid Neural Network based Feature Selection Model Improve Text Classification?cited by 1
  38. 2021Digital Twin Based Disaster Management System Proposal: DT-DMScited by 1

50 most cited of 68 All 68 (JSON)