# Efficient user history modeling with amortized inference for deep learning recommendation models

arXiv: https://arxiv.org/abs/2412.06924 · HTML: https://mlsanity.com/paper/2412.06924/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 6 of 6)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Take a Fresh Look at Recommender Systems from an Evaluation Standpoint](https://mlsanity.com/paper/2210.04149/) (2022)
- [ConvFormer: Revisiting Transformer for Sequential User Modeling](https://mlsanity.com/paper/2308.02925/) (2023)
- [Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation](https://mlsanity.com/paper/2311.01056/) (2023)

## Built on by (most cited first, 5 of 5)

- [TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation](https://mlsanity.com/paper/2506.02267/) (2025)
- [Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment](https://mlsanity.com/paper/2508.02929/) (2025)
- [UniPinRec: Unifying Generative Retrieval and Ranking at Pinterest Scale](https://mlsanity.com/paper/2606.00422/) (2026)
- [From Features to Transformers: Redefining Ranking for Scalable Impact](https://mlsanity.com/paper/2502.03417/) (2025)
- [Efficient Dataset Selection for Continual Adaptation of Generative Recommenders](https://mlsanity.com/paper/2604.07739/) (2026)

## Abstract (arXiv)

We study user history modeling via Transformer encoders in deep learning recommendation models (DLRM). Such architectures can significantly improve recommendation quality, but usually incur high latency cost necessitating infrastructure upgrades or very small Transformer models. An important part of user history modeling is early fusion of the candidate item and various methods have been studied. We revisit early fusion and compare concatenation of the candidate to each history item against appending it to the end of the list as a separate item. Using the latter method, allows us to reformulate the recently proposed amortized history inference algorithm M-FALCON citezhai2024actions for the case of DLRM models. We show via experimental results that appending with cross-attention performs on par with concatenation and that amortization significantly reduces inference costs. We conclude with results from deploying this model on the LinkedIn Feed and Ads surfaces, where amortization reduces latency by 30% compared to non-amortized inference.