# Ember: A Compiler for Efficient Embedding Operations on Decoupled Access-Execute Architectures

arXiv: https://arxiv.org/abs/2504.09870 · HTML: https://mlsanity.com/paper/2504.09870/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 65 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [The Architectural Implications of Facebook's DNN-based Personalized Recommendation](https://mlsanity.com/paper/1906.03109/) (2019)
- [RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing](https://mlsanity.com/paper/1912.12953/) (2019)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [Understanding Training Efficiency of Deep Learning Recommendation Models at Scale](https://mlsanity.com/paper/2011.05497/) (2020)
- [Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.11810/) (2019)
- [Neural Input Search for Large Scale Recommendation Models](https://mlsanity.com/paper/1907.04471/) (2019)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks](https://mlsanity.com/paper/2011.06391/) (2020)
- [The gem5 Simulator: Version 20.0+](https://mlsanity.com/paper/2007.03152/) (2020)
- [The ARM Scalable Vector Extension](https://mlsanity.com/paper/1803.06185/) (2018)
- [Looplets: A Language For Structured Coiteration](https://mlsanity.com/paper/2209.05250/) (2022)
- [UniSparse: An Intermediate Language for General Sparse Format Customization](https://mlsanity.com/paper/2403.05802/) (2024)
- [Compilation of Modular and General Sparse Workspaces](https://mlsanity.com/paper/2404.04541/) (2024)
- [Scaling up HBM Efficiency of Top-K SpMV for Approximate Embedding Similarity on FPGAs](https://mlsanity.com/paper/2103.04808/) (2021)
- [Solving Large Top-K Graph Eigenproblems with a Memory and Compute-optimized FPGA Design](https://mlsanity.com/paper/2103.10040/) (2021)
- [SpChar: Characterizing the Sparse Puzzle via Decision Trees](https://mlsanity.com/paper/2304.06944/) (2023)

## Abstract (arXiv)

Irregular embedding lookups are a critical bottleneck in recommender models, sparse large language models, and graph learning models. In this paper, we first demonstrate that, by offloading these lookups to specialized access units, Decoupled Access-Execute (DAE) processors achieve 2.6× higher performance and 6.4× higher performance/watt than GPUs on end-to-end models. Then, we propose the Ember compiler for automatically generating optimized DAE code from PyTorch and TensorFlow. Conversely from other DAE compilers, Ember features multiple intermediate representations specifically designed for different optimization levels. In this way, Ember can implement all optimizations to match the performance of hand-written code, unlocking the full potential of DAE architectures at scale.