mlsanityData to 7 Oct 2026

Automatic Pharma News Categorization

arXiv 2201.00688 ↗First posted 28 Dec 2021cs.IR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We use a text dataset consisting of 23 news categories relevant to pharma information science, in order to compare the fine-tuning performance of multiple transformer models in a classification task. Using a well-balanced dataset with multiple autoregressive and autocoding transformation models, we compare their fine-tuning performance. To validate the winning approach, we perform diagnostics of model behavior on mispredicted instances, including inspection of category-wise metrics, evaluation of prediction certainty and assessment of latent space representations. Lastly, we propose an ensemble model consisting of the top performing individual predictors and demonstrate that this approach offers a modest improvement in the F1 metric.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 28 Dec 2021.

Builds on

17 of 32 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2019Cross-lingual Language Model Pretrainingcited by 1,088
5 more
  1. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  2. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  3. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  4. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  5. 2020AutoTrans: Automating Transformer Design via Reinforced Architecture Searchcited by 12

Built on by

Not cited yet.