mlsanityData to 14 Oct 2026

polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics

arXiv 2209.14803 ↗First posted 29 Sep 2022cond-mat.mtrl-sci cs.AI cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

Polymers are a vital part of everyday life. Their chemical universe is so large that it presents unprecedented opportunities as well as significant challenges to identify suitable application-specific candidates. We present a complete end-to-end machine-driven polymer informatics pipeline that can search this space for suitable candidates at unprecedented speed and accuracy. This pipeline includes a polymer chemical fingerprinting capability called polyBERT (inspired by Natural Language Processing concepts), and a multitask learning approach that maps the polyBERT fingerprints to a host of properties. polyBERT is a chemical linguist that treats the chemical structure of polymers as a chemical language. The present approach outstrips the best presently available concepts for polymer property prediction based on handcrafted fingerprint schemes in speed by two orders of magnitude while preserving accuracy, thus making it a strong candidate for deployment in scalable architectures including cloud infrastructures.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 24 of its 73 references that are in the map. First posted on arXiv 29 Sep 2022.

Builds on

24 of 73 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  9. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  10. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  11. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  12. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
12 more
  1. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  2. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  3. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  4. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 127
  5. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  6. 2022TransPolymer: a Transformer-based language model for polymer property predictionscited by 43
  7. 2020Polymer Informatics with Multi-Task Learningcited by 29
  8. 2020Polymer Informatics: Current Status and Critical Next Stepscited by 29
  9. 2022A graph representation of molecular ensembles for polymer property predictioncited by 20
  10. 2020Polymers for Extreme Conditions Designed Using Syntax-Directed Variational Autoencoderscited by 12
  11. 2021Copolymer Informatics with Multi-Task Deep Neural Networkscited by 7
  12. 2022Bioplastic Design using Multitask Deep Neural Networkscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 12; 2025: 15; 2026: 17.

48 papers.

Sort:

By area:

  1. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 122
  2. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  3. 2022Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Modelcited by 25
  4. 2025Offline Model-Based Optimization: Comprehensive Reviewcited by 15
  5. 2024MMPolymer: A Multimodal Multitask Pretraining Framework for Polymer Property Predictioncited by 12
  6. 2024Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNNcited by 8
  7. 2025Unified Cross-Scale 3D Generation and Understanding via Autoregressive Modelingcited by 8
  8. 2024MAGE: Model-Level Graph Neural Networks Explanations via Motif-based Graph Generationcited by 7
  9. 2024PolyCL: Contrastive Learning for Polymer Representation Learning via Explicit and Implicit Augmentationscited by 7
  10. 2024Inverse Design of Copolymers Including Stoichiometry and Chain Architecturecited by 7
  11. 2023Transferring a molecular foundation model for polymer property predictionscited by 6
  12. 2026Tabular foundation models for in-context prediction of molecular propertiescited by 6
36 more
  1. 2025Multimodal machine learning with large language embedding model for polymer property predictioncited by 5
  2. 2026Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectoriecited by 5
  3. 2024Bridging AI and Science: Implications from a Large-Scale Literature Analysis of AI4Sciencecited by 3
  4. 2025PolyConf: Unlocking Polymer Conformation Generation through Hierarchical Generative Modelscited by 3
  5. 2026Polymer-Agent: Large Language Model Agent for Polymer Designcited by 3
  6. 2024Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer propertiescited by 2
  7. 2024Machine Learning Based Prediction of Proton Conductivity in Metal-Organic Frameworkscited by 2
  8. 2024Molecular topological deep learning for polymer property predictioncited by 2
  9. 2026What Does a Chemical Language Model Know About Molecules?cited by 2
  10. 2023MotifPiece: A Data-Driven Approach for Effective Motif Extraction and Molecular Representation Learningcited by 1
  11. 2024Accelerating materials discovery for polymer solar cells: Data-driven insights enabled by natural language processingcited by 1
  12. 2024MolTRES: Improving Chemical Language Representation Learning for Molecular Property Predictioncited by 1
  13. 2025Joint Embedding Predictive Architecture for self-supervised pretraining on polymer molecular graphscited by 1
  14. 2025Open Polymer Challenge: Post-Competition Reportcited by 1
  15. 2026Topology-Aware Multiscale Mixture of Experts for Efficient Molecular Property Predictioncited by 1
  16. 2026MoleCode unlocks structural intelligence in large language modelscited by 1
  17. 2026A Multi-Agent Framework for Automated Coarse-Grained Molecular Dynamics of Polymerscited by 1
  18. 2025AI-Driven Discovery of High Performance Polymer Electrodes for Next-Generation Batteriescited by 0
  19. 2025Semantic Embeddings of Chemical Elements for Enhanced Materials Inference and Discoverycited by 0
  20. 2025Multi-modal cascade feature transfer for polymer property predictioncited by 0
  21. 2025Information fusion strategy integrating pre-trained language model and contrastive learning for materials knowledge miningcited by 0
  22. 2025MIPS: a Multimodal Infinite Polymer Sequence Pre-training Framework for Polymer Property Predictioncited by 0
  23. 2025A Conformation-Centric Generative Foundation Model for Linear Polymer Modeling and Designcited by 0
  24. 2025PolyRecommender: A Multimodal Recommendation System for Polymer Discoverycited by 0
  25. 2025Fine-Tuning Vision-Language Models for Multimodal Polymer Property Predictioncited by 0
  26. 2025PolySet: Restoring the Statistical Ensemble Nature of Polymers for Machine Learningcited by 0
  27. 2026Multimodal Machine Learning for Soft High-k Elastomers under Data Scarcitycited by 0
  28. 2026PolyMon: A Unified Framework for Polymer Property Predictioncited by 0
  29. 2026It's All Connected: Topology-Aware Structural Graph Encoding Improves Performance on Polymer Predictioncited by 0
  30. 2026Towards Discovery of Polymers for Insulin Delivery via Physics-Grounded Agentic Workflowscited by 0
  31. 2026Periodic Topological Deep Learning for Polymer Design and Discoverycited by 0
  32. 2026Empowering Polymeric Materials Discovery by Artificial Intelligencecited by 0
  33. 2026UniPolymer: A Unified Framework for Property Prediction, Structure Recommendation, and Evaluation in Polyimide Designcited by 0
  34. 2026PolymerGPT: Multi-property Optimization with a Decoder-Based GPT Model for Generative Polymer Designcited by 0
  35. 2026Molecular representation shapes the balance between target fidelity and exploration in flow based polymer generationcited by 0
  36. 2026An open benchmark for machine learning-based polymer property predictioncited by 0