# High Entropy Alloy property predictions using Transformer-based language model

arXiv: https://arxiv.org/abs/2411.04861 · HTML: https://mlsanity.com/paper/2411.04861/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2024.

## Built on by (most cited first, 2 of 2)

- [Sparse mixed linear modeling with anchor-based guidance for high-entropy alloy discovery](https://mlsanity.com/paper/2504.20354/) (2025)
- [Multi-Task Learning for Metal Alloy Property Prediction: An Empirical Study of Negative Transfer and Mitigation Strategies](https://mlsanity.com/paper/2512.22740/) (2025)

## Abstract (arXiv)

This study introduces a language transformer-based machine learning model to predict key mechanical properties of high-entropy alloys (HEAs), addressing the challenges due to their complex, multi-principal element compositions and limited experimental data. By pre-training the transformer on extensive synthetic materials data and fine-tuning it with specific HEA datasets, the model effectively captures intricate elemental interactions through self-attention mechanisms. This approach mitigates data scarcity issues via transfer learning, enhancing predictive accuracy for properties like elongation (%) and ultimate tensile strength (UTS) compared to traditional regression models such as Random Forests and Gaussian Processes. The model's interpretability is enhanced by visualizing attention weights, revealing significant elemental relationships that align with known metallurgical principles. This work demonstrates the potential of transformer models to accelerate materials discovery and optimization, enabling accurate property predictions, thereby advancing the field of materials informatics.