mlsanityData to 7 Oct 2026

CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts

arXiv 2609.35883 ↗First posted 27 Sep 2026cs.CR cs.LG q-bio.GNcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Genome foundation models are growing into sparse mixture-of-experts (MoE) networks whose expert weights no longer fit on the machines that hold the sequences, yet sending a private genome to rented accelerators exposes it: we show that a single server hosting one expert recovers the input nucleotides with 99.8% top-1 accuracy. We present CipherGenome, a protocol that keeps the embedding, attention and router of a 15.1B-parameter MoE genome model on a trusted thin client and outsources every expert projection, 95.8% of the parameters, to untrusted and possibly colluding GPU servers under module-LWE encryption. The design exploits three structural facts: expert layers are linear between two SwiGLU gates, expert weights are public, and GPU integer tensor cores can evaluate a ciphertext-weight product exactly modulo 248 in a single GEMM. The client evaluates the nonlinearity exactly and re-encrypts with fresh secrets, so no polynomial approximation or bootstrapping is ever needed. On 72 windows from 12 bacterial genomes, encryption adds 2.54 × 10-4 nats per token of KL divergence (95% CI upper bound 3.95 × 10-4), below a pre-registered non-inferiority margin and indistinguishable from bf16 inference, while the same inversion attack falls to chance level. A reusable public hint cuts end-to-end latency by 3.54 times, wire compression reduces traffic 6.8 times, per-layer padding reduces routing leakage from 54.9% to 8.9% accuracy, and HE-compatible int4 experts remain non-inferior to their plaintext counterparts. Per expert and token, the server-side cost is more than six orders of magnitude below a CKKS baseline.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 28 of its 51 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

28 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale14 Jan

Other references · 27

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2024Mixtral of Expertscited by 1,785
  5. 2020GLU Variants Improve Transformercited by 1,724
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  10. 2018Gazelle: A Low Latency Framework for Secure Neural Network Inferencecited by 193
  11. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 189
  12. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
15 more
  1. 2018Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardwarecited by 101
  2. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 84
  3. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 67
  4. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 55
  5. 2021TenSEAL: A Library for Encrypted Tensor Operations Using Homomorphic Encryptioncited by 51
  6. 2023PUMA: Secure Inference of LLaMA-7B in Five Minutescited by 44
  7. 2025MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMscited by 11
  8. 2024Stealing User Prompts from Mixture of Expertscited by 10
  9. 2026Expert Selections In MoE Models Reveal (Almost) As Much As Textcited by 6
  10. 2025Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryptioncited by 5
  11. 2025CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routingcited by 4
  12. 2026SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Computecited by 1
  13. 2026How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequencescited by 1
  14. 2026Quantifying Memorization and Privacy Risks in Genomic Language Modelscited by 1
  15. 2026Feasibility of Homomorphic Inference for a Genomic Foundation Modelcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs
  2. 2024MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
  3. 2026SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
  4. 2024Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
  5. 2026Groundhog Bit-Flip Attack: Seeding Infinite Generation Loops in Mixture-of-Experts LLMs through Bit Flips
  6. 2026Eliminating Hidden Serialization in Multi-Node Megakernel Communication
  7. 2026Public-Decay Homomorphic State Space Models for Private Sequence Inference
  8. 2024LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing