# Can LLMs Compress (and Decompress)? Evaluating Code Understanding and Execution via Invertibility

arXiv: https://arxiv.org/abs/2601.13398 · HTML: https://mlsanity.com/paper/2601.13398/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Builds on (most cited first, 28 of 28)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2401.03065/) (2024)
- [OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement](https://mlsanity.com/paper/2402.14658/) (2024)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [Isambard-AI: a leadership class supercomputer optimised specifically for Artificial Intelligence](https://mlsanity.com/paper/2410.11199/) (2024)
- [CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction](https://mlsanity.com/paper/2502.07316/) (2025)
- [Reasoning Runtime Behavior of a Program with LLM: How Far Are We?](https://mlsanity.com/paper/2403.16437/) (2024)
- [Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?](https://mlsanity.com/paper/2502.19361/) (2025)
- [Forward-Backward Reasoning in Large Language Models for Mathematical Verification](https://mlsanity.com/paper/2308.07758/) (2023)
- [Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain](https://mlsanity.com/paper/2310.14053/) (2023)
- [CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2408.13001/) (2024)
- [Unsupervised Evaluation of Code LLMs with Round-Trip Correctness](https://mlsanity.com/paper/2402.08699/) (2024)
- [Unlocking Memorization in Large Language Models with Dynamic Soft Prompting](https://mlsanity.com/paper/2409.13853/) (2024)
- [Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates](https://mlsanity.com/paper/2402.05980/) (2024)
- [A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models](https://mlsanity.com/paper/2501.18482/) (2025)
- [Assessing Coherency and Consistency of Code Execution Reasoning by Large Language Models](https://mlsanity.com/paper/2510.15079/) (2025)
- [GroundCocoa: A Benchmark for Evaluating Compositional & Conditional Reasoning in Language Models](https://mlsanity.com/paper/2404.04237/) (2024)
- [Benchmarking and Understanding Compositional Relational Reasoning of LLMs](https://mlsanity.com/paper/2412.12841/) (2024)

## Built on by (most cited first, 4 of 4)

- [LLMs Corrupt Your Documents When You Delegate](https://mlsanity.com/paper/2604.15597/) (2026)
- [SchGen: PCB Schematic Generation with Semantic-Grounded Code Representations](https://mlsanity.com/paper/2605.30345/) (2026)
- [The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models](https://mlsanity.com/paper/2609.21509/) (2026)
- [Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer](https://mlsanity.com/paper/2609.31587/) (2026)

## Abstract (arXiv)

LLMs demonstrate strong performance on code benchmarks, yet consistent reasoning across forward and backward execution remains elusive. We present RoundTripCodeEval (RTCE), a benchmark of four code execution reasoning tasks that evaluates round-trip consistency through execution-free, exact-match assessment of bijection fidelity across four lossless compression algorithms. We evaluate state-of-the-art Code-LLMs under zero-shot prompting, supervised fine-tuning on execution traces, and iterative self-reflection. All approaches yield only modest improvements and none closes the gap, revealing that current LLMs lack the internal coherence required for reliable bidirectional code reasoning. RTCE surfaces findings invisible to existing benchmarks: models frequently pass individual forward and backward tasks yet fail the combined round-trip, exposing mutually inconsistent internal representations; SFT and self-reflection saturate after one revision round, indicating they cannot repair fundamental algorithmic misunderstandings; and failures persist even on simple bijections such as RLE, suggesting that algorithmic complexity is not the sole root cause.footnoteCode and dataset are available at https://github.com/Nickil21/round-trip-code-compression.