# TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest

arXiv: https://arxiv.org/abs/2306.00248 · HTML: https://mlsanity.com/paper/2306.00248/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Improved Recurrent Neural Networks for Session-based Recommendations](https://mlsanity.com/paper/1606.08117/) (2016)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)
- [Related Pins at Pinterest: The Evolution of a Real-World Recommender System](https://mlsanity.com/paper/1702.07969/) (2017)
- [Modeling Personalized Item Frequency Information for Next-basket Recommendation](https://mlsanity.com/paper/2006.00556/) (2020)
- [Rethinking Personalized Ranking at Pinterest: An End-to-End Approach](https://mlsanity.com/paper/2209.08435/) (2022)

## Built on by (most cited first, 30 of 35)

- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta](https://mlsanity.com/paper/2311.09544/) (2023)
- [Hierarchical LoRA MoE for Efficient CTR Model Scaling](https://mlsanity.com/paper/2510.10432/) (2025)
- [TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation](https://mlsanity.com/paper/2506.02267/) (2025)
- [InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction](https://mlsanity.com/paper/2411.09852/) (2024)
- [PinRec: Unified Generative Retrieval for Pinterest Recommender Systems](https://mlsanity.com/paper/2504.10507/) (2025)
- [PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform](https://mlsanity.com/paper/2507.12704/) (2025)
- [MerRec: A Large-scale Multipurpose Mercari Dataset for Consumer-to-Consumer Recommendation Systems](https://mlsanity.com/paper/2402.14230/) (2024)
- [LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation](https://mlsanity.com/paper/2601.20083/) (2026)
- [Factual and Personalized Recommendations using Language Models and Reinforcement Learning](https://mlsanity.com/paper/2310.06176/) (2023)
- [OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2504.17811/) (2025)
- [Building a Scalable, Effective, and Steerable Search and Ranking Platform](https://mlsanity.com/paper/2409.02856/) (2024)
- [Sliding Window Training -- Utilizing Historical Recommender Systems Data for Foundation Models](https://mlsanity.com/paper/2409.14517/) (2024)
- [RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems](https://mlsanity.com/paper/2508.00201/) (2025)
- [InteractRank: Personalized Web-Scale Search Pre-Ranking with Cross Interaction Features](https://mlsanity.com/paper/2504.06609/) (2025)
- [Efficient and Effective Query Context-Aware Learning-to-Rank Model for Sequential Recommendation](https://mlsanity.com/paper/2507.03789/) (2025)
- [Warmer for Less: A Cost-Efficient Strategy for Cold-Start Recommendations at Pinterest](https://mlsanity.com/paper/2512.17277/) (2025)
- [Privacy Preserving Conversion Modeling in Data Clean Room](https://mlsanity.com/paper/2505.14959/) (2025)
- [BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models](https://mlsanity.com/paper/2601.22925/) (2026)
- [UniPinRec: Unifying Generative Retrieval and Ranking at Pinterest Scale](https://mlsanity.com/paper/2606.00422/) (2026)
- [Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning](https://mlsanity.com/paper/2607.14192/) (2026)
- [PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest](https://mlsanity.com/paper/2607.22518/) (2026)
- [Cross-Domain Web Information Extraction at Pinterest](https://mlsanity.com/paper/2508.01096/) (2025)
- [CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer](https://mlsanity.com/paper/2602.11410/) (2026)
- [PRAGMA: Revolut Foundation Model](https://mlsanity.com/paper/2604.08649/) (2026)
- [UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence](https://mlsanity.com/paper/2605.09040/) (2026)
- [Context Features Are Cheap: Rank-Aware Decomposition for Efficient Feature Interaction in Recommender Systems](https://mlsanity.com/paper/2605.27450/) (2026)
- [JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling](https://mlsanity.com/paper/2606.19108/) (2026)
- [SOHET: Sequence Of Heterogeneous Events Transformer with Self-Supervised Pre-Training](https://mlsanity.com/paper/2606.21356/) (2026)
- [CMSL: Constructive Multi-Sequence Learning for Recommendation Systems](https://mlsanity.com/paper/2606.28533/) (2026)

## Abstract (arXiv)

Sequential models that encode user activity for next action prediction have become a popular design choice for building web-scale personalized recommendation systems. Traditional methods of sequential recommendation either utilize end-to-end learning on realtime user actions, or learn user representations separately in an offline batch-generated manner. This paper (1) presents Pinterest's ranking architecture for Homefeed, our personalized recommendation product and the largest engagement surface; (2) proposes TransAct, a sequential model that extracts users' short-term preferences from their realtime activities; (3) describes our hybrid approach to ranking, which combines end-to-end sequential modeling via TransAct with batch-generated user embeddings. The hybrid approach allows us to combine the advantages of responsiveness from learning directly on realtime user activity with the cost-effectiveness of batch user representations learned over a longer time period. We describe the results of ablation studies, the challenges we faced during productionization, and the outcome of an online A/B experiment, which validates the effectiveness of our hybrid ranking model. We further demonstrate the effectiveness of TransAct on other surfaces such as contextual recommendations and search. Our model has been deployed to production in Homefeed, Related Pins, Notifications, and Search at Pinterest.