# CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention

arXiv: https://arxiv.org/abs/2204.06519 · HTML: https://mlsanity.com/paper/2204.06519/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Neural Factorization Machines for Sparse Predictive Analytics](https://mlsanity.com/paper/1708.05027/) (2017)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Embarrassingly Shallow Autoencoders for Sparse Data](https://mlsanity.com/paper/1905.03375/) (2019)
- [AutoSVD++: An Efficient Hybrid Collaborative Filtering Model via Contractive Auto-encoders](https://mlsanity.com/paper/1704.00551/) (2017)
- [Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach](https://mlsanity.com/paper/1905.12862/) (2019)
- [Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers](https://mlsanity.com/paper/1905.10630/) (2019)
- [Critically Examining the Claimed Value of Convolutions over User-Item Embedding Maps for Recommender Systems](https://mlsanity.com/paper/2007.11893/) (2020)

## Built on by (most cited first, 22 of 22)

- [A Survey on User Behavior Modeling in Recommender Systems](https://mlsanity.com/paper/2302.11087/) (2023)
- [Attention Mixtures for Time-Aware Sequential Recommendation](https://mlsanity.com/paper/2304.08158/) (2023)
- [Pacer and Runner: Cooperative Learning Framework between Single- and Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2407.11245/) (2024)
- [Cracking the Code of Negative Transfer: A Cooperative Game Theoretic Approach for Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2311.13188/) (2023)
- [A Survey on Large Language Models in Multimodal Recommender Systems](https://mlsanity.com/paper/2505.09777/) (2025)
- [Retrieval of Temporal Event Sequences from Textual Descriptions](https://mlsanity.com/paper/2410.14043/) (2024)
- [Positional encoding is not the same as context: A study on positional encoding for sequential recommendation](https://mlsanity.com/paper/2405.10436/) (2024)
- [FLARE: Fusing Language Models and Collaborative Architectures for Recommender Enhancement](https://mlsanity.com/paper/2409.11699/) (2024)
- [Are EEG Sequences Time Series? EEG Classification with Time Series Models and Joint Subject Training](https://mlsanity.com/paper/2404.06966/) (2024)
- [Proxy-based Item Representation for Attribute and Context-aware Recommendation](https://mlsanity.com/paper/2312.06145/) (2023)
- [HMAR: Hierarchical Masked Attention for Multi-Behaviour Recommendation](https://mlsanity.com/paper/2405.09638/) (2024)
- [REGEN: A Dataset and Benchmarks with Natural Language Critiques and Narratives](https://mlsanity.com/paper/2503.11924/) (2025)
- [Tri-Attention: Explicit Context-Aware Attention Mechanism for Natural Language Processing](https://mlsanity.com/paper/2211.02899/) (2022)
- [BOLT: An Automated Deep Learning Framework for Training and Deploying Large-Scale Search and Recommendation Models on Commodity CPU Hardware](https://mlsanity.com/paper/2303.17727/) (2023)
- [Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition](https://mlsanity.com/paper/2305.13583/) (2023)
- [Predictive Modeling: BIM Command Recommendation Based on Large-scale Usage Logs](https://mlsanity.com/paper/2504.05319/) (2025)
- [Dynamic Temporal Positional Encodings for Early Intrusion Detection in IoT](https://mlsanity.com/paper/2506.18114/) (2025)
- [Sequential Recommendation on Temporal Proximities with Contrastive Learning and Self-Attention](https://mlsanity.com/paper/2402.09784/) (2024)
- [Cross-Representation Knowledge Transfer for Improved Sequential Recommendations](https://mlsanity.com/paper/2602.23471/) (2026)
- [Rethinking Convolutional Networks for Attribute-Aware Sequential Recommendation](https://mlsanity.com/paper/2605.04723/) (2026)
- [Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems](https://mlsanity.com/paper/2607.24865/) (2026)
- [OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval](https://mlsanity.com/paper/2607.27475/) (2026)

## Abstract (arXiv)

In sparse recommender settings, users' context and item attributes play a crucial role in deciding which items to recommend next. Despite that, recent works in sequential and time-aware recommendations usually either ignore both aspects or only consider one of them, limiting their predictive performance. In this paper, we address these limitations by proposing a context and attribute-aware recommender model (CARCA) that can capture the dynamic nature of the user profiles in terms of contextual features and item attributes via dedicated multi-head self-attention blocks that extract profile-level features and predicting item scores. Also, unlike many of the current state-of-the-art sequential item recommendation approaches that use a simple dot-product between the most recent item's latent features and the target items embeddings for scoring, CARCA uses cross-attention between all profile items and the target items to predict their final scores. This cross-attention allows CARCA to harness the correlation between old and recent items in the user profile and their influence on deciding which item to recommend next. Experiments on four real-world recommender system datasets show that the proposed model significantly outperforms all state-of-the-art models in the task of item recommendation and achieving improvements of up to 53% in Normalized Discounted Cumulative Gain (NDCG) and Hit-Ratio. Results also show that CARCA outperformed several state-of-the-art dedicated image-based recommender systems by merely utilizing image attributes extracted from a pre-trained ResNet50 in a black-box fashion.