mlsanityData to 14 Oct 2026

Addressing the Accuracy-Cost Tradeoff in Material Property Prediction: A Teacher-Student Strategy

arXiv 2309.04482 ↗First posted 22 Aug 2023cond-mat.mtrl-sci cs.LGcited by 0

On the map: Knowledge distillation → Graph neural network distillation

Abstract (arXiv)

Deep learning has revolutionized the process of new material discovery, with state-of-the-art models now able to predict material properties based solely on chemical compositions, thus eliminating the necessity for material structures. However, this cost-effective method has led to a trade-off in model accuracy. Specifically, the accuracy of Chemical Composition-based Property Prediction Models (CPMs) significantly lags behind that of Structure-based Property Prediction Models (SPMs). To tackle this challenge, we propose an innovative Teacher-Student (T-S) strategy, where a pre-trained SPM serves as the 'teacher' to enhance the accuracy of the CPM. Leveraging the T-S strategy, T-S CrabNet has risen to become the most accurate model among current CPMs. Initially, we demonstrated the universality of this strategy. On the Materials Project (MP) and Jarvis datasets, we validated the effectiveness of the T-S strategy in boosting the accuracy of CPMs with two distinct network structures, namely CrabNet and Roost. This led to CrabNet, under the guidance of the T-S strategy, emerging as the most accurate model among the current CPMs. Moreover, this strategy shows remarkable efficacy in small datasets. When predicting the formation energy on a small MP dataset comprising merely 5% of the samples, the T-S strategy boosted CrabNet's accuracy by 37.1%, exceeding the enhancement effect of the T-S strategy on the whole dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation. Builds on 10 of its 31 references that are in the map. First posted on arXiv 22 Aug 2023.

Builds on

10 of 31 references.

References · 10

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2023Segment Anythingcited by 8,234
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  8. 2019Predicting materials properties without crystal structure: Deep representation learning from stoichiometrycited by 62
  9. 2016Adapting Models to Signal Degradation using Distillationcited by 11
  10. 2023Connectivity Optimized Nested Graph Networks for Crystal Structurescited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Semi-supervised teacher-student deep neural network for materials discovery
  2. 2026DISTAL: Distillation and Self-Supervised Pretraining for Structure-Agnostic Materials Property Prediction
  3. 2022ParticleGrid: Enabling Deep Learning using 3D Representation of Materials
  4. 2025Role of Large Language Models and Retrieval-Augmented Generation for Accelerating Crystalline Material Discovery: A Systematic Review
  5. 2026Robust and Efficient AI Frameworks for Scalable Material Design and Property Prediction
  6. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools
  7. 2025MatterChat: A Multi-Modal LLM for Material Science
  8. 2025Aligning Reasoning LLMs for Materials Discovery with Physics-aware Rejection Sampling