# MD-HIT: Machine learning for materials property prediction with dataset redundancy control

arXiv: https://arxiv.org/abs/2307.04351 · HTML: https://mlsanity.com/paper/2307.04351/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 24 references that are in the map.
- First posted on arXiv 10 Jul 2023.

## Builds on (most cited first, 3 of 3)

- [Predicting materials properties without crystal structure: Deep representation learning from stoichiometry](https://mlsanity.com/paper/1910.00617/) (2019)
- [Machine learning modeling of superconducting critical temperature](https://mlsanity.com/paper/1709.02727/) (2017)
- [Scalable deeper graph neural networks for high-performance materials property prediction](https://mlsanity.com/paper/2109.12283/) (2021)

## Abstract (arXiv)

Materials datasets are usually featured by the existence of many redundant (highly similar) materials due to the tinkering material design practice over the history of materials research. For example, the materials project database has many perovskite cubic structure materials similar to SrTiO3. This sample redundancy within the dataset makes the random splitting of machine learning model evaluation to fail so that the ML models tend to achieve over-estimated predictive performance which is misleading for the materials science community. This issue is well known in the field of bioinformatics for protein function prediction, in which a redundancy reduction procedure (CD-Hit) is always applied to reduce the sample redundancy by ensuring no pair of samples has a sequence similarity greater than a given threshold. This paper surveys the overestimated ML performance in the literature for both composition based and structure based material property prediction. We then propose a material dataset redundancy reduction algorithm called MD-HIT and evaluate it with several composition and structure based distance threshold sfor reducing data set sample redundancy. We show that with this control, the predicted performance tends to better reflect their true prediction capability. Our MD-hit code can be freely accessed at https://github.com/usccolumbia/MD-HIT