RailX: A Flexible, Scalable, and Low-Cost Network Architecture for Hyper-Scale LLM Training Systems

Increasingly large AI workloads are calling for hyper-scale infrastructure; however, traditional interconnection network architecture is neither scalable nor cost-effective enough. Tree-based topologies such as the textitRail-optimized network are extremely expensive, while direct topologies such as textitTorus have insufficient bisection bandwidth and flexibility. In this paper, we propose textitRailX, a reconfigurable network architecture based on intra-node direct connectivity and inter-node circuit switching. Nodes and optical switches are physically 2D-organized, achieving better scalability than existing centralized circuit switching networks. We propose a novel interconnection method based on textitHamiltonian Decomposition theory to organize separate rail-based rings into textitall-to-all topology, simultaneously optimizing ring-collective and all-to-all communication. More than 100K chips with hyper bandwidth can be interconnected with a flat switching layer, and the diameter is only 2~4 inter-node hops. The network cost per injection/All-Reduce bandwidth of textitRailX is less than 10% of the Fat-Tree, and the cost per bisection/All-to-All bandwidth is less than 50% of the Fat-Tree. Specifically, only ~1.3B is required to interconnect 200K chips with 1.8TB bandwidth. textitRailX can also be used in the ML-as-a-service (MLaaS) scenario, where single or multiple training workloads with various shapes, scales, and parallelism strategies can be flexibly mapped, and failures can be worked around.