mlsanityData to 7 Oct 2026

In-Context Learning for Few-Shot Molecular Property Prediction

arXiv 2310.08863 ↗First posted 13 Oct 2023cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

In-context learning has become an important approach for few-shot learning in Large Language Models because of its ability to rapidly adapt to new tasks without fine-tuning model parameters. However, it is restricted to applications in natural language and inapplicable to other domains. In this paper, we adapt the concepts underpinning in-context learning to develop a new algorithm for few-shot molecular property prediction. Our approach learns to predict molecular properties from a context of (molecule, property measurement) pairs and rapidly adapts to new properties without fine-tuning. On the FS-Mol and BACE molecular property prediction benchmarks, we find this method surpasses the performance of recent meta-learning algorithms at small support sizes and is competitive with the best methods at large support sizes.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 45 of its 69 references that are in the map. First posted on arXiv 13 Oct 2023.

Builds on

45 of 69 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  12. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
33 more
  1. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  2. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  3. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  4. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  5. 2019How Can We Know What Language Models Know?cited by 953
  6. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  7. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  8. 2020A Generalization of Transformer Networks to Graphscited by 653
  9. 2021AST: Audio Spectrogram Transformercited by 634
  10. 2014Neural Turing Machinescited by 630
  11. 2021MetaICL: Learning to Learn In Contextcited by 450
  12. 2018Conditional Neural Processescited by 421
  13. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  14. 2021True Few-Shot Learning with Language Modelscited by 343
  15. 2017Meta Networkscited by 268
  16. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  17. 2022Structure-Aware Transformer for Graph Representation Learningcited by 191
  18. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  19. 2021Meta-learning via Language Model In-context Tuningcited by 163
  20. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  21. 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
  22. 2021GraphiT: Encoding Graph Structure in Transformerscited by 146
  23. 2019Improved Few-Shot Visual Classificationcited by 114
  24. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  25. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  26. 2020Molecule Attention Transformercited by 92
  27. 2019SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discoverycited by 89
  28. 2017Learning to Remember Rare Eventscited by 63
  29. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  30. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  31. 2023Context-enriched molecule representations improve few-shot drug discoverycited by 23
  32. 2022Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Predictioncited by 21
  33. 2020Improving Few-Shot Learning through Multi-task Representation Learning Theorycited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 2.

7 papers.

  1. 2025Training a Scientific Reasoning Model for Chemistrycited by 54
  2. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  3. 2023Context-Aware Meta-Learningcited by 20
  4. 2024Unsupervised Meta-Learning via In-Context Learningcited by 3
  5. 2025Language Models for Controllable DNA Sequence Designcited by 1
  6. 2026AbICL: In-Context Learning for Antigen-Specific Antibody Affinity Rankingcited by 0
  7. 2026CalibHyper: Chance-Corrected Relational Hypergraphs for Few-Shot Molecular Property Predictioncited by 0