mlsanityData to 7 Oct 2026

Crystal Twins: Self-supervised Learning for Crystalline Material Property Prediction

arXiv 2205.01893 ↗First posted 4 May 2022cs.LG cond-mat.mtrl-scicited by 34

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) models have been widely successful in the prediction of material properties. However, large labeled datasets required for training accurate ML models are elusive and computationally expensive to generate. Recent advances in Self-Supervised Learning (SSL) frameworks capable of training ML models on unlabeled data have mitigated this problem and demonstrated superior performance in computer vision and natural language processing tasks. Drawing inspiration from the developments in SSL, we introduce Crystal Twins (CT): an SSL method for crystalline materials property prediction. Using a large unlabeled dataset, we pre-train a Graph Neural Network (GNN) by applying the redundancy reduction principle to the graph latent embeddings of augmented instances obtained from the same crystalline system. By sharing the pre-trained weights when fine-tuning the GNN for regression tasks, we significantly improve the performance for 7 challenging material property prediction benchmarks

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 28 of its 72 references that are in the map. First posted on arXiv 4 May 2022.

Builds on

28 of 72 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  8. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
16 more
  1. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  2. 2020Directional Message Passing for Molecular Graphscited by 556
  3. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  4. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 268
  5. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 254
  6. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
  7. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  8. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  9. 2022Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrastcited by 37
  10. 2020Orbital Graph Convolutional Neural Network for Material Property Predictioncited by 29
  11. 2021Dual-view Molecule Pre-trainingcited by 27
  12. 2020Self-Supervised Graph Transformer on Large-Scale Molecular Datacited by 17
  13. 2019Self-Supervised Dialogue Learningcited by 16
  14. 2021AugLiChem: Data Augmentation Library of Chemical Structures for Machine Learningcited by 10
  15. 2020CRYSPNet: Crystal Structure Predictions via Neural Networkcited by 7
  16. 2022Formula graph self-attention network for representation-domain independent materials discoverycited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 5; 2024: 9; 2025: 9; 2026: 6.

34 papers.

Sort:

  1. 2022TransPolymer: a Transformer-based language model for polymer property predictionscited by 43
  2. 2022MOFormer: Self-Supervised Transformer model for Metal-Organic Framework Property Predictioncited by 28
  3. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  4. 2022Graph Neural Networks for Moleculescited by 21
  5. 2022Capturing long-range interaction with reciprocal space neural networkcited by 13
  6. 2023GPT-MolBERTa: GPT Molecular Features Language Model for molecular property predictioncited by 13
  7. 2023Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentialscited by 11
  8. 2023Accelerating Molecular Graph Neural Networks via Knowledge Distillationcited by 11
  9. 2025MOFGPT: Generative Design of Metal-Organic Frameworks using Language Modelscited by 11
  10. 2024AI-driven inverse design of materials: Past, present and futurecited by 10
  11. 2025Siamese Foundation Models for Crystal Structure Predictioncited by 8
  12. 2023Materials Informatics Transformer: A Language Model for Interpretable Materials Properties Predictioncited by 7
22 more
  1. 2024AlloyBERT: Alloy Property Prediction with Large Language Modelscited by 5
  2. 2026Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectoriecited by 5
  3. 2023A Crystal-Specific Pre-Training Framework for Crystal Material Property Predictioncited by 3
  4. 2024Self-supervised learning for crystal property prediction via denoisingcited by 3
  5. 2024CrysAtom: Distributed Representation of Atoms for Crystal Property Predictioncited by 3
  6. 2025CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learningcited by 3
  7. 2025Leveraging neural network interatomic potentials for a foundation model of chemistrycited by 3
  8. 2026Self-Conditioned Denoising for Atomistic Representation Learningcited by 3
  9. 2024Enhancing material property prediction with ensemble deep graph convolutional networkscited by 2
  10. 2025Beyond Force Metrics: Pre-Training MLFFs for Stable MD Simulationscited by 2
  11. 2022Predicting CO2 Absorption in Ionic Liquids with Molecular Descriptors and Explainable Graph Neural Networkscited by 1
  12. 2025Contrastive Learning of English Language and Crystal Graphs for Multimodal Representation of Materials Knowledgecited by 1
  13. 2025Supervised Pretraining for Material Property Predictioncited by 1
  14. 2026A New Workflow for Materials Discovery Bridging the Gap Between Experimental Databases and Graph Neural Networkscited by 1
  15. 2026From Natural Language to Materials Discovery:The Materials Knowledge Navigation Agentcited by 1
  16. 2026TriForces: Augmenting Atomistic GNNs for Transferable Representationscited by 1
  17. 2024Accelerating Material Property Prediction using Generically Complete Isometry Invariantscited by 0
  18. 2024Pretraining Strategy for Neural Potentialscited by 0
  19. 2024Establishing Deep InfoMax as an effective self-supervised learning methodology in materials informaticscited by 0
  20. 2025MoMa: A Modular Deep Learning Framework for Material Property Predictioncited by 0
  21. 2025Leveraging Large Language Models to Address Data Scarcity in Machine Learning: Applications in Graphene Synthesiscited by 0
  22. 2026Model Agnostic Graph Prompt Learning for Crystal Property Predictioncited by 0