# Measuring and Reducing Cross-Vendor Mismatch in Language Models

arXiv: https://arxiv.org/abs/2610.05458 · HTML: https://mlsanity.com/paper/2610.05458/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [OLMoE: Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2409.02060/) (2024)
- [ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic](https://mlsanity.com/paper/2402.12840/) (2024)
- [Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers](https://mlsanity.com/paper/2510.11370/) (2025)
- [Defeating the Training-Inference Mismatch via FP16](https://mlsanity.com/paper/2510.26788/) (2025)
- [LLM-42: Enabling Determinism in LLM Inference with Verified Speculation](https://mlsanity.com/paper/2601.17768/) (2026)
- [Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch](https://mlsanity.com/paper/2511.17826/) (2025)
- [Log Probability Tracking of LLM APIs](https://mlsanity.com/paper/2512.03816/) (2025)
- [Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It](https://mlsanity.com/paper/2602.01826/) (2026)
- [Diagnosing Training Inference Mismatch in LLM Reinforcement Learning via a Zero-Mismatch Reference](https://mlsanity.com/paper/2605.14220/) (2026)
- [Testing GPU Numerics: Finding Numerical Differences Between NVIDIA and AMD GPUs](https://mlsanity.com/paper/2410.09172/) (2024)
- [MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs](https://mlsanity.com/paper/2509.11662/) (2025)
- [MarginGate: Sparse Margin-Triggered Verification for Batch-Invariant LLM Inference](https://mlsanity.com/paper/2605.30218/) (2026)
- [FTTN: Feature-Targeted Testing for Numerical Properties of NVIDIA & AMD Matrix Accelerators](https://mlsanity.com/paper/2403.00232/) (2024)
- [Jais 2: A Family of Arabic-Centric Open Large Language Models](https://mlsanity.com/paper/2608.13580/) (2026)
- [Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification](https://mlsanity.com/paper/2609.11356/) (2026)
- [Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures](https://mlsanity.com/paper/2609.25624/) (2026)

## Abstract (arXiv)

Running the same language model on different graphics processing unit (GPU) vendors can produce different logits, even when the model weights and inputs are the same. We analyze cross-vendor mismatch in two dense and two mixture-of-experts (MoE) models with five metric families, namely bitwise equality, logit differences, top-K consistency, token agreement, and task accuracy. We trace one source of the mismatch to accumulation order inside vendors' matrix instructions. Upcasting to FP32 reduces the dense model's logit error by 43% at three times the runtime, yet keeping only the MLPs in BF16 retains 94% of this gain at 1.3 times the runtime, so most of the cost of full upcasting buys little. In the MoE models, FP32 and FP16 both lower the probability error but raise the logit error and change expert selection, and FP16 fails in the dense model. An output-head low-rank adapter (LoRA) does not help either, since the final hidden state does not predict the mismatch. The mismatch also carries into training. With every seed fixed, a student distilled from a teacher running on AMD answers 431 MMLU questions differently from one distilled from the same teacher on NVIDIA. Under FP32 upcasting, bitwise equality barely changes while the output distributions move most of the way to the reference, so judging cross-vendor agreement by a single measure misreads both its cost and its gains. Code is available at https://github.com/crova-project/crova.