mlsanityData to 7 Oct 2026

Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferral

arXiv 2502.12701 ↗First posted 18 Feb 2025cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Larger models often outperform smaller ones but come with high computational costs. Cascading offers a potential solution. By default, it uses smaller models and defers only some instances to larger, more powerful models. However, designing effective deferral rules remains a challenge. In this paper, we propose a simple yet effective approach for machine translation, using existing quality estimation (QE) metrics as deferral rules. We show that QE-based deferral allows a cascaded system to match the performance of a larger model while invoking it for a small fraction (30% to 50%) of the examples, significantly reducing computational costs. We validate this approach through both automatic and human evaluation.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

32 of 50 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  7. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  8. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  9. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  10. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  11. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  12. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 266
20 more
  1. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 210
  2. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  3. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  4. 2023Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawscited by 155
  5. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 115
  6. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  7. 2024EuroLLM: Multilingual Language Models for Europecited by 103
  8. 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 94
  9. 2024MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Taskcited by 92
  10. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  11. 2017Curriculum Learning and Minibatch Bucketing in Neural Machine Translationcited by 63
  12. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  13. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 57
  14. 2022Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systemscited by 43
  15. 2024Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selectioncited by 32
  16. 2024Can Automatic Metrics Assess High-Quality Translations?cited by 16
  17. 2024Mitigating Metric Bias in Minimum Bayes Risk Decodingcited by 15
  18. 2022TangoBERT: Reducing Inference Cost by using Cascaded Architecturecited by 13
  19. 2022Exact Paired-Permutation Testing for Structured Test Statisticscited by 6
  20. 2024Reranking Laws for Language Generation: A Communication-Theoretic Perspectivecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 16
  2. 2025Early-Exit and Instant Confidence Translation Quality Estimationcited by 2
  3. 2026RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deploymentcited by 2
  4. 2026Cluster, Route, Escalate: Cascaded Framework for Cost-Aware LLM Servingcited by 0