mlsanityData to 7 Oct 2026

Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks

arXiv 2210.13979 ↗First posted 22 Oct 2022cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large pretrained Transformer-based language models like BERT and GPT have changed the landscape of Natural Language Processing (NLP). However, fine tuning such models still requires a large number of training examples for each target task, thus annotating multiple datasets and training these models on various downstream tasks becomes time consuming and expensive. In this work, we propose a simple extension of the Prototypical Networks for few-shot text classification. Our main idea is to replace the class prototypes by Gaussians and introduce a regularization term that encourages the examples to be clustered near the appropriate class centroids. Experimental results show that our method outperforms various strong baselines on 13 public and 4 internal datasets. Furthermore, we use the class distributions as a tool for detecting potential out-of-distribution (OOD) data points during deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 22 Oct 2022.

Builds on

25 of 46 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2019Publicly Available Clinical BERT Embeddingscited by 995
  10. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  11. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  12. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
13 more
  1. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  2. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  3. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
  4. 2013Fast Computation of Wasserstein Barycenterscited by 216
  5. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  6. 2019Learning and Evaluating General Linguistic Intelligencecited by 78
  7. 2019Induction Networks for Few-Shot Text Classificationcited by 64
  8. 2019Learning to Few-Shot Learn Across Diverse Natural Language Classification Taskscited by 57
  9. 2020Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Taskscited by 51
  10. 2021Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descentcited by 32
  11. 2022Few-shot learning for medical text: A systematic reviewcited by 12
  12. 2022Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representationcited by 8
  13. 2022Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reportscited by 1

Built on by

Not cited yet.