Sequence modeling of higher-order wave modes of binary black hole mergers

Higher-order gravitational wave modes from quasi-circular, spinning, non-precessing binary black hole mergers encode key information about these systems' nonlinear dynamics. We model these waveforms using transformer architectures, targeting the evolution from late inspiral through ringdown. Our data is derived from the textttNRHybSur3dq8 surrogate model, which includes spherical harmonic modes up to ℓ ≤ 4 (excluding (4,0), (4,±1) and including (5,5) modes). These waveforms span mass ratios q ≤ 8, spin components sz_1,2 ∈ [-0.8, 0.8], and inclination angles θ ∈ [0, π]. The model processes input data over the time interval t ∈ [-5000textrmM, -100textrmM) and generates predictions for the plus and cross polarizations, (h+, h×), over the interval t ∈ [-100textrmM, 130textrmM]. Utilizing 16 NVIDIA A100 GPUs on the Delta supercomputer, we trained the transformer model in 15 hours on over 14 million samples. The model's performance was evaluated on a test dataset of 840,000 samples, achieving mean and median overlap scores of 0.996 and 0.997, respectively, relative to the surrogate-based ground truth signals. We further benchmark the model on numerical relativity waveforms from the SXS catalog, finding that it generalizes well to out-of-distribution systems, capable of reproducing the dynamics of systems with mass ratios up to q=15 and spin magnitudes up to 0.998, with a median overlap of 0.969 across 521 NR waveforms and up to 0.998 in face-on/off configurations. These results demonstrate that transformer-based models can capture the nonlinear dynamics of binary black hole mergers with high accuracy, even outside the surrogate training domain, enabling fast sequence modeling of higher-order wave modes.