# CTRL: Connect Collaborative and Language Model for CTR Prediction

arXiv: https://arxiv.org/abs/2306.02841 · HTML: https://mlsanity.com/paper/2306.02841/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2023.

## Built on by (most cited first, 14 of 14)

- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [CoIR: A Comprehensive Benchmark for Code Information Retrieval Models](https://mlsanity.com/paper/2407.02883/) (2024)
- [Language-Based User Profiles for Recommendation](https://mlsanity.com/paper/2402.15623/) (2024)
- [LettinGo: Explore User Profile Generation for Recommendation System](https://mlsanity.com/paper/2506.18309/) (2025)
- [End-to-end Training for Recommendation with Language-based User Profiles](https://mlsanity.com/paper/2410.18870/) (2024)
- [Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers](https://mlsanity.com/paper/2406.05255/) (2024)
- [Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Models](https://mlsanity.com/paper/2412.04107/) (2024)
- [Field Matters: A Lightweight LLM-enhanced Method for CTR Prediction](https://mlsanity.com/paper/2505.14057/) (2025)
- [CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction](https://mlsanity.com/paper/2508.03668/) (2025)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Learning Multi-Branch Cooperation for Enhanced Click-Through Rate Prediction at Taobao](https://mlsanity.com/paper/2411.13057/) (2024)
- [Reasoning-guided Collaborative Filtering with Language Models for Explainable Recommendation](https://mlsanity.com/paper/2602.05544/) (2026)
- [Enhancing Serendipity Recommendation System by Constructing Dynamic User Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2508.04032/) (2025)
- [SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation](https://mlsanity.com/paper/2607.24025/) (2026)

## Abstract (arXiv)

Traditional click-through rate (CTR) prediction models convert the tabular data into one-hot vectors and leverage the collaborative relations among features for inferring the user's preference over items. This modeling paradigm discards essential semantic information. Though some works like P5 and CTR-BERT have explored the potential of using Pre-trained Language Models (PLMs) to extract semantic signals for CTR prediction, they are computationally expensive and suffer from low efficiency. Besides, the beneficial collaborative relations are not considered, hindering the recommendation performance. To solve these problems, in this paper, we propose a novel framework CTRL, which is industrial-friendly and model-agnostic with superior inference efficiency. Specifically, the original tabular data is first converted into textual data. Both tabular data and converted textual data are regarded as two different modalities and are separately fed into the collaborative CTR model and pre-trained language model. A cross-modal knowledge alignment procedure is performed to fine-grained align and integrate the collaborative and semantic signals, and the lightweight collaborative model can be deployed online for efficient serving after fine-tuned with supervised signals. Experimental results on three public datasets show that CTRL outperforms the state-of-the-art (SOTA) CTR models significantly. Moreover, we further verify its effectiveness on a large-scale industrial recommender system.