# CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation

arXiv: https://arxiv.org/abs/2504.15254 · HTML: https://mlsanity.com/paper/2504.15254/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 21 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems](https://mlsanity.com/paper/2306.03091/) (2023)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation](https://mlsanity.com/paper/2308.01861/) (2023)
- [Multi-lingual Evaluation of Code Generation Models](https://mlsanity.com/paper/2210.14868/) (2022)
- [CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings](https://mlsanity.com/paper/2501.01257/) (2025)
- [DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories](https://mlsanity.com/paper/2405.19856/) (2024)
- [Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study](https://mlsanity.com/paper/2403.08604/) (2024)
- [Towards Translating Real-World Code with LLMs: A Study of Translating to Rust](https://mlsanity.com/paper/2405.11514/) (2024)
- [Measuring The Impact Of Programming Language Distribution](https://mlsanity.com/paper/2302.01973/) (2023)
- [VERT: Verified Equivalent Rust Transpilation with Large Language Models as Few-Shot Learners](https://mlsanity.com/paper/2404.18852/) (2024)
- [C2SaferRust: Transforming C Projects into Safer Rust with NeuroSymbolic Techniques](https://mlsanity.com/paper/2501.14257/) (2025)
- [Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysis](https://mlsanity.com/paper/2412.14234/) (2024)
- [Ownership guided C to Rust translation](https://mlsanity.com/paper/2303.10515/) (2023)
- [TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills](https://mlsanity.com/paper/2306.07285/) (2023)

## Built on by (most cited first, 19 of 19)

- [EvoC2Rust: A Skeleton-guided Framework for Project-Level C-to-Rust Translation](https://mlsanity.com/paper/2508.04295/) (2025)
- [A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System](https://mlsanity.com/paper/2510.09721/) (2025)
- [A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression](https://mlsanity.com/paper/2604.19572/) (2026)
- [SETA: Scaling Environments for Terminal Agents](https://mlsanity.com/paper/2607.10891/) (2026)
- [OSVBench: Benchmarking LLMs on Specification Generation Tasks for Operating System Verification](https://mlsanity.com/paper/2504.20964/) (2025)
- [SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration?](https://mlsanity.com/paper/2608.23564/) (2026)
- [MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair](https://mlsanity.com/paper/2509.16187/) (2025)
- [Adversarial Agent Collaboration for Correctness Improvements of C to Safe Rust Translation](https://mlsanity.com/paper/2510.03879/) (2025)
- [Project-Level C-to-Rust Translation via Pointer Knowledge Graphs](https://mlsanity.com/paper/2510.10956/) (2025)
- [ReCodeAgent: A Multi-agent Workflow for Language-Agnostic Translation and Validation of Large-Scale Repositories](https://mlsanity.com/paper/2604.07341/) (2026)
- [ORBIT: Guided Agentic Orchestration for Autonomous C-to-Rust Transpilation](https://mlsanity.com/paper/2604.12048/) (2026)
- [RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilities](https://mlsanity.com/paper/2607.04729/) (2026)
- [SoK: Understanding (New) Security Issues Across AI4Code Use Cases](https://mlsanity.com/paper/2512.18456/) (2025)
- [CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language](https://mlsanity.com/paper/2603.14501/) (2026)
- [Documentation-Guided Agentic Codebase Migration from C to Rust](https://mlsanity.com/paper/2605.14634/) (2026)
- [CNnotator: LLM-Guided Memory Safety Annotation Synthesis](https://mlsanity.com/paper/2606.21822/) (2026)
- [Generative Compilation: On-the-Fly Compiler Feedback as AI Generates Code](https://mlsanity.com/paper/2607.13921/) (2026)
- [Fine-Tuning Qwen3-27B for C-to-Rust Code Translation: A Three-Stage Curriculum of Pretraining, Debugging-Aware SFT, and Task-Specific SFT](https://mlsanity.com/paper/2608.13681/) (2026)
- [Translator vs. Challenger: Adversarial Agentic Learning for C-to-Rust Translation](https://mlsanity.com/paper/2609.15381/) (2026)

## Abstract (arXiv)

C-to-Rust transpilation is essential for modernizing legacy C code while enhancing safety and interoperability with modern Rust ecosystems. However, no dataset currently exists for evaluating whether a system can transpile C into safe Rust that passes a set of test cases. We introduce CRUST-Bench, a dataset of 100 C repositories, each paired with manually-written interfaces in safe Rust as well as test cases that can be used to validate correctness of the transpilation. By considering entire repositories rather than isolated functions, CRUST-Bench captures the challenges of translating complex projects with dependencies across multiple files. The provided Rust interfaces provide explicit specifications that ensure adherence to idiomatic, memory-safe Rust patterns, while the accompanying test cases enforce functional correctness. We evaluate state-of-the-art large language models (LLMs) on this task and find that safe and idiomatic Rust generation is still a challenging problem for various state-of-the-art methods and techniques. We also provide insights into the errors LLMs usually make in transpiling code from C to safe Rust. The best performing model, OpenAI o1, is able to solve only 15 tasks in a single-shot setting. Improvements on CRUST-Bench would lead to improved transpilation systems that can reason about complex scenarios and help in migrating legacy codebases from C into languages like Rust that ensure memory safety. You can find the dataset and code at https://github.com/anirudhkhatry/CRUST-bench.