Quantum Attention by Overlap Interference: Predicting Classical and Many-Body Quantum Sequences

We propose a variational quantum implementation of self-attention (QSA)-the core operation in transformers and large language models-which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and a degree-k polynomial kernel, and estimates a loss based on R'enyi-1/2 entropic functionals via two observables' expectation values, avoiding the decoding of amplitude-encoded predictions into classical probabilities. QSA also accommodates a constrained, trainable data-embedding tying state overlaps to data-level similarities. Its dominant end-to-end training complexity scales as Oleft(μ-1k2Tdright), versus Oleft(T dk+1right) of the fairest classical comparison, with μ a training signal; we show numerically that this allows a complexity advantage in the regime where sequence length T dominates the embedding size d. In simulations, our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising trajectories-establishing trainable attention as a practical primitive for quantum dynamical modeling.