# Productively Deploying Emerging Models on Emerging Platforms: A Top-Down Approach for Testing and Debugging

arXiv: https://arxiv.org/abs/2404.09151 · HTML: https://mlsanity.com/paper/2404.09151/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 68 references that are in the map.
- First posted on arXiv 14 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models](https://mlsanity.com/paper/2212.14834/) (2022)
- [MLIR: A Compiler Infrastructure for the End of Moore's Law](https://mlsanity.com/paper/2002.11054/) (2020)
- [NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers](https://mlsanity.com/paper/2207.13066/) (2022)
- [Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source](https://mlsanity.com/paper/2201.06589/) (2022)
- [DocTer: Documentation Guided Fuzzing for Testing Deep Learning API Functions](https://mlsanity.com/paper/2109.01002/) (2021)
- [Fuzzing Automatic Differentiation in Deep-Learning Libraries](https://mlsanity.com/paper/2302.04351/) (2023)
- [Fuzzing Deep-Learning Libraries via Automated Relational API Inference](https://mlsanity.com/paper/2207.05531/) (2022)
- [Relax: Composable Abstractions for End-to-End Dynamic Machine Learning](https://mlsanity.com/paper/2311.02103/) (2023)
- [NeuRI: Diversifying DNN Generation via Inductive Rule Inference](https://mlsanity.com/paper/2302.02261/) (2023)
- [TorchProbe: Fuzzing Dynamic Deep Learning Compilers](https://mlsanity.com/paper/2310.20078/) (2023)

## Built on by (most cited first, 2 of 2)

- [Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference](https://mlsanity.com/paper/2606.02430/) (2026)
- [MANTRA: a Framework for Multi-stage Adaptive Noise TReAtment During Training](https://mlsanity.com/paper/2512.04319/) (2025)

## Abstract (arXiv)

While existing machine learning (ML) frameworks focus on established platforms, like running CUDA on server-grade GPUs, there have been growing demands to enable emerging AI applications in a broader set of scenarios, such as running Large Language Models (LLMs) within browsers and mobile phones. However, deploying emerging models on new platforms (such as Metal and WebGPU) presents significant software engineering challenges due to rapid model evolution and limited tooling and practices for these platforms. Previous practice for ML model deployment often follows a bottom-up fashion, where engineers first implement individual required operators and then put them together. However, this traditional development approach fails to meet the productivity requirements when deploying emerging ML applications, with the testing and debugging part as a bottleneck. To this end, we introduce textscTapML, a top-down approach designed to streamline model deployment on diverse platforms. While the traditional bottom-up approach requires crafting manual tests, textscTapML automatically creates high-quality, realistic test data through operator-wise test carving. Furthermore, textscTapML uses a migration-based strategy to gradually offload model implementation from the mature source platform to the target platform, minimizing the debugging scope of compound errors. textscTapML has been used as the default development method in the MLC-LLM project to deploy emerging ML models. Within 2 years, textscTapML has accelerated the deployment of 105 emerging models in 27 model architectures across 5 emerging platforms. We show that textscTapML effectively boosts developer productivity while ensuring the quality of deployed models. Furthermore, we summarize comprehensive case studies from our real-world development, offering best practices for developing emerging ML systems.