mlsanityData to 14 Oct 2026

Enriching Biomedical Knowledge for Low-resource Language Through Large-Scale Translation

arXiv 2210.05598 ↗First posted 11 Oct 2022cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Biomedical data and benchmarks are highly valuable yet very limited in low-resource languages other than English such as Vietnamese. In this paper, we make use of a state-of-the-art translation model in English-Vietnamese to translate and produce both pretrained as well as supervised data in the biomedical domains. Thanks to such large-scale translation, we introduce ViPubmedT5, a pretrained Encoder-Decoder Transformer model trained on 20 million translated abstracts from the high-quality public PubMed corpus. ViPubMedT5 demonstrates state-of-the-art results on two different biomedical benchmarks in summarization and acronym disambiguation. Further, we release ViMedNLI - a new NLP task in Vietnamese translated from MedNLI using the recently public En-vi translation model and carefully refined by human experts, with evaluations of existing methods against ViPubmedT5.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

33 of 54 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  10. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
21 more
  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  2. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  3. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  4. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
  5. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 299
  6. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  7. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  8. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 151
  9. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 149
  10. 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
  11. 2020CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characterscited by 87
  12. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  13. 2018A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literaturecited by 79
  14. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  15. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  16. 2021Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processingcited by 57
  17. 2022ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generationcited by 37
  18. 2021PhoMT: A High-Quality and Large-Scale Benchmark Dataset for Vietnamese-English Machine Translationcited by 12
  19. 2019The Source-Target Domain Mismatch Problem in Machine Translationcited by 8
  20. 2022Geographical Distance Is The New Hyperparameter: A Case Study Of Finding The Optimal Pre-trained Language For English-isiZulu Machine Translationcited by 5
  21. 2019Facebook AI's WAT19 Myanmar-English Translation Task Submissioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Taskscited by 36
  2. 2024Real-time Speech Summarization for Medical Conversationscited by 2
  3. 2025Comparing Large Language Models and Traditional Machine Translation Tools for Translating Medical Consultation Summaries: A Pilot Studycited by 1
  4. 2026Blinded Radiologist and LLM-Based Evaluation of LLM-Generated Japanese Translations of Chest CT Reports: Comparative Studycited by 0