Revealing the Phase Diagram of Kitaev Materials by Machine Learning: Cooperation and Competition between Spin Liquids

Kitaev materials are promising materials for hosting quantum spin liquids and investigating the interplay of topological and symmetry-breaking phases. We use an unsupervised and interpretable machine-learning method, the tensorial-kernel support vector machine, to study the honeycomb Kitaev-Gamma model in a magnetic field. Our machine learns the global classical phase diagram and the associated analytical order parameters, including several distinct spin liquids, two exotic S3 magnets, and two modulated S3 × Z3 magnets. We find that the extension of Kitaev spin liquids and a field-induced suppression of magnetic order already occur in the large-S limit, implying that critical parts of the physics of Kitaev materials can be understood at the classical level. Moreover, the two S3 × Z3 orders are induced by competition between Kitaev and Gamma spin liquids and feature a different type of spin-lattice entangled modulation, which requires a matrix description instead of scalar phase factors. Our work provides a direct instance of a machine detecting new phases and paves the way towards the development of automated tools to explore unsolved problems in many-body physics.