mlsanityData to 14 Oct 2026

Multi-Item-Query Attention for Stable Sequential Recommendation

arXiv 2509.24424 ↗First posted 29 Sep 2025cs.IR cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The inherent instability and noise in user interaction data challenge sequential recommendation systems. Prevailing masked attention models, relying on a single query from the most recent item, are sensitive to this noise, reducing prediction reliability. We propose the Multi-Item-Query attention mechanism (MIQ-Attn) to enhance model stability and accuracy. MIQ-Attn constructs multiple diverse query vectors from user interactions, effectively mitigating noise and improving consistency. It is designed for easy adoption as a drop-in replacement for existing single-query attention. Experiments show MIQ-Attn significantly improves performance on benchmark datasets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

25 of 33 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  4. 2018Self-Attentive Sequential Recommendationcited by 1,184
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  6. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  7. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  8. 2015Image-based Recommendations on Styles and Substitutescited by 706
  9. 2015Session-based Recommendations with Recurrent Neural Networkscited by 695
  10. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  11. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
  12. 2018Session-based Recommendation with Graph Neural Networkscited by 312
13 more
  1. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  2. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 188
  3. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  4. 2022Intent Contrastive Learning for Sequential Recommendationcited by 91
  5. 2017Joint Deep Modeling of Users and Items Using Reviews for Recommendationcited by 84
  6. 2023Debiased Contrastive Learning for Sequential Recommendationcited by 38
  7. 2023Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?cited by 30
  8. 2021Modeling Sequences as Distributions with Uncertainty for Sequential Recommendationcited by 22
  9. 2022A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendationcited by 22
  10. 2018Next Item Recommendation with Self-Attentioncited by 15
  11. 2020Contrastive Learning for Sequential Recommendationcited by 13
  12. 2022Effective and Efficient Training for Sequential Recommendation using Recency Samplingcited by 13
  13. 2023Improving Sequential Recommendation Models with an Enhanced Loss Functioncited by 1

Built on by

Not cited yet.