mlsanityData to 7 Oct 2026

FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference

arXiv 2101.05615 ↗First posted 13 Jan 2021cs.LG cs.PFcited by 40

Not on the map. Why?

Abstract (arXiv)

Deep learning models typically use single-precision (FP32) floating point data types for representing activations and weights, but a slew of recent research work has shown that computations with reduced-precision data types (FP16, 16-bit integers, 8-bit integers or even 4- or 2-bit integers) are enough to achieve same accuracy as FP32 and are much more efficient. Therefore, we designed fbgemm, a high-performance kernel library, from ground up to perform high-performance quantized inference on current generation CPUs. fbgemm achieves efficiency by fusing common quantization operations with a high-performance gemm implementation and by shape- and size-specific kernel code generation at runtime. The library has been deployed at Facebook, where it delivers greater than 2x performance gains with respect to our current production baseline.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 1 of its 11 references that are in the map. First posted on arXiv 13 Jan 2021.

Builds on

1 of 11 references.

References · 1

  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 10; 2024: 4; 2025: 5; 2026: 9.

40 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  3. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 184
  4. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 112
  5. 2023Stable and low-precision training for large-scale vision-language modelscited by 74
  6. 2021DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memoriescited by 42
  7. 2021Torch.fx: Practical Program Capture and Transformation for Deep Learning in Pythoncited by 38
  8. 2022AutoShard: Automated Embedding Table Sharding for Recommender Systemscited by 17
  9. 2022DreamShard: Generalizable Embedding Table Placement for Recommender Systemscited by 14
  10. 2023Revisiting Neural Retrieval on Acceleratorscited by 14
  11. 2023MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendationcited by 9
  12. 2026Towards Automated Kernel Generation in the Era of LLMscited by 9
28 more
  1. 2023FPUS23: An Ultrasound Fetus Phantom Dataset with Deep Neural Network Evaluations for Fetus Orientations, Fetal Planes, and Anatomical Featurescited by 8
  2. 2025PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platformcited by 8
  3. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7
  4. 2022A Practical Stereo Depth System for Smart Glassescited by 7
  5. 2021Model Pruning Based on Quantified Similarity of Feature Mapscited by 6
  6. 2024Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendationcited by 6
  7. 2021Supporting Massive DLRM Inference Through Software Defined Memorycited by 5
  8. 2023Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Modelscited by 5
  9. 2025μnit Scaling: Simple and Scalable FP8 LLM Trainingcited by 5
  10. 2026An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inferencecited by 3
  11. 2021The NiuTrans System for the WMT21 Efficiency Taskcited by 2
  12. 2022Verifiable and Energy Efficient Medical Image Analysis with Quantised Self-attentive Deep Neural Networkscited by 2
  13. 2025Two-dimensional Sparse Parallelism for Large Scale Deep Learning Recommendation Model Trainingcited by 2
  14. 2023Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environmentscited by 1
  15. 2024Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platformscited by 1
  16. 2025Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updatescited by 1
  17. 2026LoKA: Low-precision Kernel Applications for Recommendation Models At Scalecited by 1
  18. 2022FullPack: Full Vector Utilization for Sub-Byte Quantized Inference on General Purpose CPUscited by 0
  19. 2023HEAT: A Highly Efficient and Affordable Training System for Collaborative Filtering Based Recommendation on CPUscited by 0
  20. 2023MTrainS: Improving DLRM training efficiency using heterogeneous memoriescited by 0
  21. 2023Neural Network Compression using Binarization and Few Full-Precision Weightscited by 0
  22. 2025Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery Marketplacecited by 0
  23. 2026At the Edge of the Heart: ULP FPGA-Based CNN for On-Device Cardiac Feature Extraction in Smart Health Sensors for Astronautscited by 0
  24. 2026When KV Meets Embeddings: Dynamic GPU Memory Allocation for Accelerating Generative Recommender Servingcited by 0
  25. 2026QuantSR+: Pushing the Limit of Quantized Image Super-Resolution Networkscited by 0
  26. 2026Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGAcited by 0
  27. 2026Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Acceleratorscited by 0
  28. 2026RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Modelscited by 0