mlsanityData to 14 Oct 2026

DimGrow: Memory-Efficient Field-level Embedding Dimension Search

arXiv 2505.12683 ↗First posted 19 May 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

30 of 46 references.

References · 30

  1. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  3. 2016Wide & Deep Learning for Recommender Systemscited by 747
  4. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 373
  5. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
  6. 2019Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systemscited by 68
  7. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  8. 2020AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendationscited by 47
  9. 2021Learnable Embedding Sizes for Recommender Systemscited by 42
  10. 2019Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systemscited by 41
  11. 2019Neural Input Search for Large Scale Recommendation Modelscited by 39
  12. 2020Learning to Embed Categorical Features without Embedding Tables for Recommendationcited by 38
18 more
  1. 2020Model Size Reduction Using Frequency Based Double Hashing for Recommender Systemscited by 33
  2. 2022AutoField: Automating Feature Selection in Deep Recommender Systemscited by 30
  3. 2020Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systemscited by 24
  4. 2019Post-Training 4-bit Quantization on Embedding Tablescited by 22
  5. 2021Learning Elastic Embeddings for Customizing On-Device Recommenderscited by 19
  6. 2020Mixed-Precision Embedding Using a Cachecited by 18
  7. 2022Serving Deep Learning Models with Deduplication from Relational Databasescited by 15
  8. 2022OptEmbed: Learning Optimal Embedding Table for Click-through Rate Predictioncited by 14
  9. 2022Learning Compressed Embeddings for On-Device Inferencecited by 13
  10. 2022Single-shot Embedding Dimension Search in Recommender Systemcited by 13
  11. 2021Learning Effective and Efficient Embedding via an Adaptively-Masked Twins-based Layercited by 12
  12. 2023Experimental Analysis of Large-scale Learnable Vector Storage Compressioncited by 12
  13. 2021Binary Code based Hash Embedding for Web-scale Applicationscited by 11
  14. 2022Adaptive Low-Precision Training for Embeddings in Click-Through Rate Predictioncited by 11
  15. 2024ERASE: Benchmarking Feature Selection Methods for Deep Recommender Systemscited by 9
  16. 2023Clustered Embedding Learning for Recommender Systemscited by 8
  17. 2023SHARK: A Lightweight Model Compression Approach for Large-scale Recommender Systemscited by 6
  18. 2022Field-wise Embedding Size Search via Structural Hard Auxiliary Mask Pruning for Click-Through Rate Predictioncited by 2

Built on by

Not cited yet.