# TypeT5: Seq2seq Type Inference using Static Analysis

arXiv: https://arxiv.org/abs/2303.09564 · HTML: https://mlsanity.com/paper/2303.09564/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [LambdaNet: Probabilistic Type Inference using Graph Neural Networks](https://mlsanity.com/paper/2005.02161/) (2020)
- [TypeWriter: Neural Type Prediction with Search-based Validation](https://mlsanity.com/paper/1912.03768/) (2019)
- [Learning to Update Natural Language Comments Based on Code Changes](https://mlsanity.com/paper/2004.12169/) (2020)
- [Typilus: Neural Type Hints](https://mlsanity.com/paper/2004.10657/) (2020)
- [Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python](https://mlsanity.com/paper/2101.04470/) (2021)
- [OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints](https://mlsanity.com/paper/2004.00348/) (2020)
- [Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for Python](https://mlsanity.com/paper/2105.03595/) (2021)
- [ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference](https://mlsanity.com/paper/2104.04706/) (2021)
- [Cross-Domain Evaluation of a Deep Learning-Based Type Inference System](https://mlsanity.com/paper/2208.09189/) (2022)

## Built on by (most cited first, 12 of 12)

- [CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2401.03065/) (2024)
- [EffiBench: Benchmarking the Efficiency of Automatically Generated Code](https://mlsanity.com/paper/2402.02037/) (2024)
- [Bias Testing and Mitigation in LLM-based Code Generation](https://mlsanity.com/paper/2309.14345/) (2023)
- [EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization](https://mlsanity.com/paper/2405.15189/) (2024)
- [EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2505.13004/) (2025)
- [MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation](https://mlsanity.com/paper/2405.11430/) (2024)
- [EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning](https://mlsanity.com/paper/2410.10209/) (2024)
- [Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing](https://mlsanity.com/paper/2305.18584/) (2023)
- [Beyond the Prompt: An Empirical Study of Cursor Rules](https://mlsanity.com/paper/2512.18925/) (2025)
- [The Power of Types: Exploring the Impact of Type Checking on Neural Bug Detection in Dynamically Typed Languages](https://mlsanity.com/paper/2411.15368/) (2024)
- [Learning Type Inference for Enhanced Dataflow Analysis](https://mlsanity.com/paper/2310.00673/) (2023)
- [TypyBench: Evaluating LLM Type Inference for Untyped Python Repositories](https://mlsanity.com/paper/2507.22086/) (2025)

## Abstract (arXiv)

There has been growing interest in automatically predicting missing type annotations in programs written in Python and JavaScript. While prior methods have achieved impressive accuracy when predicting the most common types, they often perform poorly on rare or complex types. In this paper, we present a new type inference method that treats type prediction as a code infilling task by leveraging CodeT5, a state-of-the-art seq2seq pre-trained language model for code. Our method uses static analysis to construct dynamic contexts for each code element whose type signature is to be predicted by the model. We also propose an iterative decoding scheme that incorporates previous type predictions in the model's input context, allowing information exchange between related code elements. Our evaluation shows that the proposed approach, TypeT5, not only achieves a higher overall accuracy (particularly on rare and complex types) but also produces more coherent results with fewer type errors -- while enabling easy user intervention.