mlsanityData to 7 Oct 2026

Universal Language Model Fine-tuning for Text Classification

arXiv 1801.06146 ↗First posted 18 Jan 2018cs.CL cs.LG stat.MLcited by 50

Not on the map. Why?

Abstract (arXiv)

Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model. Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets. Furthermore, with only 100 labeled examples, it matches the performance of training from scratch on 100x more data. We open-source our pretrained models and code.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 5; 2023: 7; 2024: 11; 2025: 6; 2026: 5.

50 papers.

Sort:

By area:

  1. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  2. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  3. 2022RoentGen: Vision-Language Foundation Model for Chest X-ray Generationcited by 89
  4. 2023How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Taskscited by 58
  5. 2021Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translationcited by 47
  6. 2024Learning to Maximize Mutual Information for Chain-of-Thought Distillationcited by 23
  7. 2021Relative Molecule Self-Attention Transformercited by 21
  8. 2021Person Re-Identification with a Locally Aware Transformercited by 19
  9. 2023Mixed Distillation Helps Smaller Language Model Better Reasoningcited by 18
  10. 2021Attention-based Clinical Note Summarizationcited by 16
  11. 2021Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivitycited by 11
  12. 2021LU-BZU at SemEval-2021 Task 2: Word2Vec and Lemma2Vec performance in Arabic Word-in-Context disambiguationcited by 9
38 more
  1. 2021Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hatecited by 8
  2. 2025Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitationscited by 6
  3. 2025NANOGPT: A Query-Driven Large Language Model Retrieval-Augmented Generation System for Nanotechnology Researchcited by 6
  4. 2021ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Modelscited by 5
  5. 2022Pre-training Transformers for Molecular Property Prediction Using Reaction Predictioncited by 5
  6. 2024Best Practices for Large Language Models in Radiologycited by 5
  7. 2025Don't Think Twice! Over-Reasoning Impairs Confidence Calibrationcited by 5
  8. 2023Investigating the Role of Feed-Forward Networks in Transformers Using Parallel Attention and Feed-Forward Net Designcited by 4
  9. 2024Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformerscited by 4
  10. 2024Modality Translation for Object Detection Adaptation Without Forgetting Prior Knowledgecited by 4
  11. 2025Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUscited by 4
  12. 2021Ceasing hate withMoH: Hate Speech Detection in Hindi-English Code-Switched Languagecited by 3
  13. 2024Reaction-conditioned De Novo Enzyme Design with GENzymecited by 3
  14. 2024Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Predictioncited by 3
  15. 2024HindiLLM: Large Language Model for Hindicited by 3
  16. 2021Comparing the Performance of NLP Toolkits and Evaluation measures in Legal Techcited by 2
  17. 2021Legal Search in Case Law and Statute Lawcited by 2
  18. 2023Low Resource Summarization using Pre-trained Language Modelscited by 2
  19. 2021Understanding Transformers for Bot Detection in Twittercited by 1
  20. 2021Zhestyatsky at SemEval-2021 Task 2: ReLU over Cosine Similarity for BERT Fine-tuningcited by 1
  21. 2021Neural Language Models with Distant Supervision to Identify Major Depressive Disorder from Clinical Notescited by 1
  22. 2021YES SIR!Optimizing Semantic Space of Negatives with Self-Involvement Rankercited by 1
  23. 2021Pretrained Transformers for Offensive Language Identification in Tanglishcited by 1
  24. 2023Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskritcited by 1
  25. 2024No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Textscited by 1
  26. 2024Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipelinecited by 1
  27. 2026Mitigating Forgetting in Continual Learning with Selective Gradient Projectioncited by 1
  28. 2026Learning to Edit Knowledge via Instruction-based Chain-of-Thought Promptingcited by 1
  29. 2022Bridging the Domain Gap for Stance Detection for the Zulu languagecited by 0
  30. 2022Multilingual Transformer Language Model for Speech Recognition in Low-resource Languagescited by 0
  31. 2022Maximizing Use-Case Specificity through Precision Model Tuningcited by 0
  32. 2024Leveraging LLMs for Predictive Insights in Food Policy and Behavioral Interventionscited by 0
  33. 2024Optimizing Speech-Input Length for Speaker-Independent Depression Classificationcited by 0
  34. 2025SemSketches-2021: experimenting with the machine processing of the pilot semantic sketches corpuscited by 0
  35. 2025Fine-tuning of Large Language Models for Constituency Parsing Using a Sequence to Sequence Approachcited by 0
  36. 2026Continually self-improving AIcited by 0
  37. 2026PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Promptscited by 0
  38. 2026Greedy dynamical meta-learningcited by 0