ESTS at WMT26: Routing-Informed Expert Pruning for Model Compression
Not on the map. Why?
Abstract (arXiv)
We describe six submissions under the team name ESTS to the unconstrained WMT26 Model Compression Shared Task for English--Simplified Chinese and English--Egyptian Arabic. We submit three compression operating points per translation direction, all derived from GPT-OSS-20B. We use task-specific routing mass to rank experts and cross-lingual routing divergence to allocate retained capacity across layers, then physically remove low-importance experts. The resulting specialists are recovery-tuned on GPT-5.1-generated synthetic translation data and further compressed by applying MXFP4 quantization to the retained expert projection weights. We additionally implement a robust inference system for the instruction-conditioned WMT26 setting, including category inference, output validation, retries, segmented fallback, and source-owned JSON reconstruction. Across our six submissions, parameter counts range from 4.186B to 7.770B and packed artifact sizes from 4.55 to 6.33~GiB. Internal xCOMET-XL evaluation using GPT-5.1 pseudo-references provides an internal comparison across the submitted compression operating points.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 15 references that are in the map. First posted on arXiv 11 Sep 2026.
Builds on
6 of 15 references.
References · 6
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
- 2025Multilingual Routing in Mixture-of-Expertscited by 29
- 2024Mapping the Podcast Ecosystem with the Structured Podcast Research Corpuscited by 6
- 2025Iterative Layer Pruning for Efficient Translation Inferencecited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Compression is Routing: Reconstruction Error as an Intrinsic Signal for Modular Language Models
- 2025CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
- 2025A Systematic Study of Compression Ordering for Large Language Models
- 2023The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models
- 2024Mixture Compressor for Mixture-of-Experts LLMs Gains More
- 2024Aggressive Post-Training Compression on Extremely Large Language Models
- 2026Depth-Aware Sensitivity Analysis of Mixture-of-Experts Models via Magnitude-Based Expert Masking
- 2026Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts