mlsanityData to 14 Oct 2026

Differentiable Weightless Neural Networks

arXiv 2410.11112 ↗First posted 14 Oct 2024cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 21 of its 50 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

21 of 50 references.

References · 21

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  4. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  5. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  6. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  7. 2021Training Neural Networks with Fixed Sparse Maskscited by 214
  8. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
  9. 2018Fast inference of deep neural networks in FPGAs for particle physicscited by 133
  10. 2021MLPerf Tiny Benchmarkcited by 124
  11. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  12. 2022Deep Differentiable Logic Gate Networkscited by 67
9 more
  1. 2020LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applicationscited by 57
  2. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 54
  3. 2020Adaptive Gradient Quantization for Data-Parallel SGDcited by 45
  4. 2019ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detectioncited by 44
  5. 2023PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inferencecited by 30
  6. 2024NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functionscited by 21
  7. 2018Joint Binary Neural Network for Multi-label Learning with Applications to Emotion Classificationcited by 12
  8. 2023ULEEN: A Novel Architecture for Ultra Low-Energy Edge Neural Networkscited by 8
  9. 2022BiFSMN: Binary Neural Network for Keyword Spottingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 16.

28 papers.

Sort:

  1. 2024Convolutional Differentiable Logic Gate Networkscited by 37
  2. 2025NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inferencecited by 14
  3. 2025Light Differentiable Logic Gate Networkscited by 9
  4. 2025WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tablescited by 6
  5. 2025LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deploymentcited by 6
  6. 2025Scalable Interconnect Learning in Boolean Networkscited by 4
  7. 2025LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neuronscited by 3
  8. 2026WARP Logic Neural Networkscited by 2
  9. 2025AI Compute Architecture and Evolution Trendscited by 1
  10. 2025Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Controlcited by 1
  11. 2026Inter-patient ECG Arrhythmia Classification with LGNs and LUTNscited by 1
  12. 2026Continuous-Flow Data-Rate-Aware CNN Inference on FPGAcited by 1
16 more
  1. 2026Learning Compact Boolean Networkscited by 1
  2. 2026BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networkscited by 1
  3. 2026HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inferencecited by 1
  4. 2026FPGN: Redefining Ultra-Fast Programmable Gate-based Neural Acceleration with Differentiable LUTscited by 1
  5. 2024Shrinking the Giant : Quasi-Weightless Transformers for Low Energy Inferencecited by 0
  6. 2025Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applicationscited by 0
  7. 2025BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Trainingcited by 0
  8. 2026Enhancing LUT-based Deep Neural Networks Inference through Architecture and Connectivity Optimizationcited by 0
  9. 2026TT-Sparse: Learning Sparse Rule Models with Differentiable Truth Tablescited by 0
  10. 2026GLANCE: Gaze-Led Attention Network for Compressed Edge-inferencecited by 0
  11. 2026From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flipscited by 0
  12. 2026Fitting Multilinear Polynomials for Logic Gate Networkscited by 0
  13. 2026CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAscited by 0
  14. 2026DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivitycited by 0
  15. 2026Task-Aware Discretization of Differentiable Logic Gate Networkscited by 0
  16. 2026Overcoming Kernel Redundancy for Scaling Logic Gate Networkscited by 0