mlsanityData to 7 Oct 2026

MoEC: Mixture of Experts Implicit Neural Compression

arXiv 2312.01361 ↗First posted 3 Dec 2023cs.CV cs.LG eess.IVcited by 4

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

Emerging Implicit Neural Representation (INR) is a promising data compression technique, which represents the data using the parameters of a Deep Neural Network (DNN). Existing methods manually partition a complex scene into local regions and overfit the INRs into those regions. However, manually designing the partition scheme for a complex scene is very challenging and fails to jointly learn the partition and INRs. To solve the problem, we propose MoEC, a novel implicit neural compression method based on the theory of mixture of experts. Specifically, we use a gating network to automatically assign a specific INR to a 3D point in the scene. The gating network is trained jointly with the INRs of different local regions. Compared with block-wise and tree-structured partitions, our learnable partition can adaptively find the optimal partition in an end-to-end manner. We conduct detailed experiments on massive and diverse biomedical data to demonstrate the advantages of MoEC against existing approaches. In most of experiment settings, we have achieved state-of-the-art results. Especially in cases of extreme compression ratios, such as 6000x, we are able to uphold the PSNR of 48.16.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 27 of its 42 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

27 of 42 references.

References · 27

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  8. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  9. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  11. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  12. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
15 more
  1. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  4. 2021A White Paper on Neural Network Quantizationcited by 453
  5. 2018DVC: An End-to-end Deep Video Compression Frameworkcited by 324
  6. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
  7. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  8. 2021NeRV: Neural Representations for Videoscited by 255
  9. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 199
  10. 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
  11. 2021Implicit Neural Representations for Image Compressioncited by 161
  12. 2022Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstructioncited by 92
  13. 2021Implicit Neural Video Compressioncited by 52
  14. 2022TINC: Tree-structured Implicit Neural Compressioncited by 14
  15. 2022SCI: A Spectrum Concentrated Implicit Neural Compression for Biomedical Datacited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024Decomposing the Neurons: Activation Sparsity via Mixture of Experts for Continual Test Time Adaptationcited by 10
  2. 2024Neural Experts: Mixture of Experts for Implicit Neural Representationscited by 9
  3. 2024Implicit Neural Image Field for Biological Microscopy Image Compressioncited by 0
  4. 2026MoE-dqINR: A Unified Mixture-of-Experts Implicit Neural Representation Framework for Scan-Specific Dynamic and Quantitative MRI Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mixture of Experts with Soft Nearest Neighbor Loss: Resolving Expert Collapse via Representation Disentanglement
  2. 2022Neural Implicit Dictionary via Mixture-of-Expert Training
  3. 2026Mixture-of-Experts-based Entropy Model for Learned Image Compression
  4. 2025MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
  5. 2022Towards Understanding Mixture of Experts in Deep Learning
  6. 2026MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts
  7. 2024SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
  8. 2026Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks