# ERASE: EaRly bAckpropagation SchEdule for Faster Training of Modern Recommendation Systems

arXiv: https://arxiv.org/abs/2608.18469 · HTML: https://mlsanity.com/paper/2608.18469/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [RankMixer: Scaling Up Ranking Models in Industrial Recommenders](https://mlsanity.com/paper/2507.15551/) (2025)
- [Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading](https://mlsanity.com/paper/2607.08964/) (2026)
- [Layer Collaboration in the Forward-Forward Algorithm](https://mlsanity.com/paper/2305.12393/) (2023)
- [Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism](https://mlsanity.com/paper/2010.08899/) (2020)
- [LiRank: Industrial Large Scale Ranking Models at LinkedIn](https://mlsanity.com/paper/2402.06859/) (2024)
- [DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation](https://mlsanity.com/paper/2506.14202/) (2025)
- [Understanding Political Polarization via Jointly Modeling Users, Connections and Multimodal Contents on Heterogeneous Graphs](https://mlsanity.com/paper/2201.05946/) (2022)
- [ASI-Bench: At the Dawn of Artificial Superintelligence](https://mlsanity.com/paper/2608.17271/) (2026)
- [Boosting Performance of Iterative Applications on GPUs: Kernel Batching with CUDA Graphs](https://mlsanity.com/paper/2501.09398/) (2025)
- [PyGraph: Robust Compiler Support for CUDA Graphs in PyTorch](https://mlsanity.com/paper/2503.19779/) (2025)
- [Inference for Balance in Dynamic Signed Networks](https://mlsanity.com/paper/2606.08786/) (2026)

## Built on by (most cited first, 2 of 2)

- [LLM Evaluation on Unseen Questions: Contextual Multidimensional IRT Model](https://mlsanity.com/paper/2608.22295/) (2026)
- [Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation](https://mlsanity.com/paper/2609.11115/) (2026)

## Abstract (arXiv)

Lightweight proxy models enable rapid experimentation without repeatedly training frontier-scale systems, but their small kernels often leave modern accelerators underutilized. Conventional training compounds this inefficiency by scheduling the forward and backward passes as disjoint phases, so spare capacity in one cannot be filled by work from the other. We reinterpret the detachment mechanism of Forward-Forward (FF) as a scheduling primitive: given a local objective, detaching a block's output removes downstream gradient dependencies, making its backward pass ready when its forward pass finishes. ERASE launches each detached subgraph's backward pass early on a separate CUDA stream, overlapping it with subsequent forward work. Execution trace on a lightweight transformer demonstrates this overlap and its limit: a kernel that saturates the device leaves no capacity for concurrency. On a large-scale click-through-rate model, detaching six dense subarchitectures improves training throughput by up to 9.51% while keeping normalized entropy close to the baseline.