# Workload-Driven Optimization for On-Device Real-Time Subtitle Translation

arXiv: https://arxiv.org/abs/2607.09957 · HTML: https://mlsanity.com/paper/2607.09957/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Builds on (most cited first, 8 of 8)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Better than Average: Paired Evaluation of NLP Systems](https://mlsanity.com/paper/2110.10746/) (2021)
- [An Analysis of BPE Vocabulary Trimming in Neural Machine Translation](https://mlsanity.com/paper/2404.00397/) (2024)
- [Mediocrity is the key for LLM as a Judge Anchor Selection](https://mlsanity.com/paper/2603.16848/) (2026)
- [NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs](https://mlsanity.com/paper/2511.07003/) (2025)
- [A Case Study on Contextual Machine Translation in a Professional Scenario of Subtitling](https://mlsanity.com/paper/2407.00108/) (2024)

## Abstract (arXiv)

This report studies on-device English-to-Traditional-Chinese subtitle translation for Taiwan under short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints. These conditions limit the value of optimizations designed for long-context or high-throughput language-model serving. Starting from LMT-60-0.6B, preliminary profiling suggests that vocabulary projection becomes a more important decode-time cost after GGUF quantization reduces the relative cost of Transformer blocks. We replace the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer, migrate the embedding space, and adapt the model through embedding calibration followed by full supervised fine-tuning. On an OpenSubtitles2024 test set, LocalSubs achieves a 59.2% tie-excluded win rate against Google Translate under GPT-4o pairwise judging. Performance is strongest on short cues and declines as cue length increases. In a separate preliminary Apple M2 Metal profiling run, LocalSubs shows a 1.63x speedup over a 151k-vocabulary baseline. The code is available on https://github.com/aiden1020/localsubs .