mlsanityData to 14 Oct 2026

ESTS at WMT26: Routing-Informed Expert Pruning for Model Compression

arXiv 2609.12310 ↗First posted 11 Sep 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We describe six submissions under the team name ESTS to the unconstrained WMT26 Model Compression Shared Task for English--Simplified Chinese and English--Egyptian Arabic. We submit three compression operating points per translation direction, all derived from GPT-OSS-20B. We use task-specific routing mass to rank experts and cross-lingual routing divergence to allocate retained capacity across layers, then physically remove low-importance experts. The resulting specialists are recovery-tuned on GPT-5.1-generated synthetic translation data and further compressed by applying MXFP4 quantization to the retained expert projection weights. We additionally implement a robust inference system for the instruction-conditioned WMT26 setting, including category inference, output validation, retries, segmented fallback, and source-owned JSON reconstruction. Across our six submissions, parameter counts range from 4.186B to 7.770B and packed artifact sizes from 4.55 to 6.33~GiB. Internal xCOMET-XL evaluation using GPT-5.1 pseudo-references provides an internal comparison across the submitted compression operating points.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 15 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

6 of 15 references.

References · 6

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  3. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  4. 2025Multilingual Routing in Mixture-of-Expertscited by 29
  5. 2024Mapping the Podcast Ecosystem with the Structured Podcast Research Corpuscited by 6
  6. 2025Iterative Layer Pruning for Efficient Translation Inferencecited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Compression is Routing: Reconstruction Error as an Intrinsic Signal for Modular Language Models
  2. 2025CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
  3. 2025A Systematic Study of Compression Ordering for Large Language Models
  4. 2023The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models
  5. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains More
  6. 2024Aggressive Post-Training Compression on Extremely Large Language Models
  7. 2026Depth-Aware Sensitivity Analysis of Mixture-of-Experts Models via Magnitude-Based Expert Masking
  8. 2026Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts