Automatic Annotation Augmentation Boosts Translation between Molecules and Natural Language
Not on the map. Why?
Abstract (arXiv)
Recent advancements in AI for biological research focus on integrating molecular data with natural language to accelerate drug discovery. However, the scarcity of high-quality annotations limits progress in this area. This paper introduces LA3, a Language-based Automatic Annotation Augmentation framework that leverages large language models to augment existing datasets, thereby improving AI training. We demonstrate the effectiveness of LA3 by creating an enhanced dataset, LaChEBI-20, where we systematically rewrite the annotations of molecules from an established dataset. These rewritten annotations preserve essential molecular information while providing more varied sentence structures and vocabulary. Using LaChEBI-20, we train LaMolT5 based on a benchmark architecture to learn the mapping between molecular representations and augmented annotations. Experimental results on text-based *de novo* molecule generation and molecule captioning demonstrate that LaMolT5 outperforms state-of-the-art models. Notably, incorporating LA3 leads to improvements of up to 301% over the benchmark architecture. Furthermore, we validate the effectiveness of LA3 notable applications in *image*, *text* and *graph* tasks, affirming its versatility and utility.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 53 references that are in the map. First posted on arXiv 10 Feb 2025.
Builds on
29 of 53 references.
References · 29
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
17 more
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
- 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
- 2020Transformers as Soft Reasoners over Languagecited by 270
- 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
- 2018Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discoverycited by 194
- 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
- 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
- 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 93
- 2023Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspectivecited by 92
- 2023MolXPT: Wrapping Molecules with Text for Generative Pre-trainingcited by 81
- 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
- 2024Text-Guided Molecule Generation with Diffusion Language Modelcited by 31
- 2024Benchmarking Large Language Models for Molecule Prediction Taskscited by 17
- 2024Harnessing Large Language Models as Post-hoc Correctorscited by 7
Built on by
Not cited yet.