mlsanityData to 7 Oct 2026

How to Fine-Tune BERT for Text Classification?

arXiv 1905.05583 ↗First posted 14 May 2019cs.CLcited by 409

Not on the map. Why?

Abstract (arXiv)

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets.

Key facts. Cited by 409 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 104; 2022: 82; 2023: 78; 2024: 73; 2025: 54; 2026: 18.

409 papers.

Sort:

By area:

  1. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  2. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  3. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  4. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  5. 2023Continual Pre-training of Language Modelscited by 194
  6. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
  7. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  8. 2023DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomecited by 139
  9. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
  10. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  11. 2023Text Classification via Large Language Modelscited by 104
  12. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
38 more
  1. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  2. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  3. 2022The Limits of Word Level Differential Privacycited by 63
  4. 2021CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systemscited by 60
  5. 2021Revisiting Uncertainty-based Query Strategies for Active Learning with Transformerscited by 59
  6. 2023How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Taskscited by 58
  7. 2022BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learningcited by 56
  8. 2022PLM-ICD: Automatic ICD Coding with Pretrained Language Modelscited by 49
  9. 2021AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuningcited by 48
  10. 2022Revisiting Transformer-based Models for Long Document Classificationcited by 48
  11. 2022NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Bettercited by 43
  12. 2021Self-Tuning for Data-Efficient Deep Learningcited by 39
  13. 2021Improved Text Classification via Contrastive Adversarial Trainingcited by 38
  14. 2021MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Educationcited by 37
  15. 2021Scaling Federated Learning for Fine-tuning of Large Language Modelscited by 36
  16. 2024KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Modelscited by 36
  17. 2023Open, Closed, or Small Language Models for Text Classification?cited by 34
  18. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  19. 2022Continual Training of Language Models for Few-Shot Learningcited by 33
  20. 2023Understanding Telecom Language Through Large Language Modelscited by 32
  21. 2023GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learningcited by 32
  22. 2025Idiosyncrasies in Large Language Modelscited by 32
  23. 2022CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domaincited by 31
  24. 2021MUDES: Multilingual Detection of Offensive Spanscited by 30
  25. 2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 30
  26. 2022Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learningcited by 30
  27. 2023MasakhaNEWS: News Topic Classification for African languagescited by 30
  28. 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
  29. 2023Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Modelscited by 29
  30. 2024"My agent understands me better": Integrating Dynamic Human-like Memory Recall and Consolidation in LLM-Based Agentscited by 29
  31. 2021L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Datasetcited by 28
  32. 2023Using Sequences of Life-events to Predict Human Livescited by 27
  33. 2025RewardAnything: Generalizable Principle-Following Reward Modelscited by 27
  34. 2021On the Utility of Gradient Compression in Distributed Training Systemscited by 26
  35. 2023Mask-guided BERT for Few Shot Text Classificationcited by 25
  36. 2021Benchmarking Multimodal AutoML for Tabular Data with Text Fieldscited by 24
  37. 2022FedAdapter: Efficient Federated Learning for Modern NLPcited by 24
  38. 2023Is ChatGPT a Good Personality Recognizer? A Preliminary Studycited by 24

50 most cited of 409 All 409 (JSON)