# Typilus: Neural Type Hints

arXiv: https://arxiv.org/abs/2004.10657 · HTML: https://mlsanity.com/paper/2004.10657/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 29 of 29)

- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [Self-Supervised Bug Detection and Repair](https://mlsanity.com/paper/2105.12787/) (2021)
- [NatGen: Generative pre-training by "Naturalizing" source code](https://mlsanity.com/paper/2206.07585/) (2022)
- [Can LLMs Replace Manual Annotation of Software Engineering Artifacts?](https://mlsanity.com/paper/2408.05534/) (2024)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python](https://mlsanity.com/paper/2101.04470/) (2021)
- [GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search](https://mlsanity.com/paper/2111.02671/) (2021)
- [On Distribution Shift in Learning-based Bug Detectors](https://mlsanity.com/paper/2204.10049/) (2022)
- [Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates](https://mlsanity.com/paper/2402.05980/) (2024)
- [TypeT5: Seq2seq Type Inference using Static Analysis](https://mlsanity.com/paper/2303.09564/) (2023)
- [AutoPruner: Transformer-Based Call Graph Pruning](https://mlsanity.com/paper/2209.03230/) (2022)
- [ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference](https://mlsanity.com/paper/2104.04706/) (2021)
- [LExecutor: Learning-Guided Execution](https://mlsanity.com/paper/2302.02343/) (2023)
- [Learning Program Semantics with Code Representations: An Empirical Study](https://mlsanity.com/paper/2203.11790/) (2022)
- [Learning Program Representations with a Tree-Structured Transformer](https://mlsanity.com/paper/2208.08643/) (2022)
- [Type Prediction With Program Decomposition and Fill-in-the-Type Training](https://mlsanity.com/paper/2305.17145/) (2023)
- [An Unbiased Transformer Source Code Learning with Semantic Vulnerability Graph](https://mlsanity.com/paper/2304.11072/) (2023)
- [PSIMiner: A Tool for Mining Rich Abstract Syntax Trees from Code](https://mlsanity.com/paper/2103.12778/) (2021)
- [Cross-Lingual Transfer Learning for Statistical Type Inference](https://mlsanity.com/paper/2107.00157/) (2021)
- [JEMMA: An Extensible Java Dataset for ML4Code Applications](https://mlsanity.com/paper/2212.09132/) (2022)
- [Treefix: Enabling Execution with a Tree of Prefixes](https://mlsanity.com/paper/2501.12339/) (2025)
- [Unveiling Project-Specific Bias in Neural Code Models](https://mlsanity.com/paper/2201.07381/) (2022)
- [Evaluating the Impact of Source Code Parsers on ML4SE Models](https://mlsanity.com/paper/2206.08713/) (2022)
- [Cross-Domain Evaluation of a Deep Learning-Based Type Inference System](https://mlsanity.com/paper/2208.09189/) (2022)
- [Implant Global and Local Hierarchy Information to Sequence based Code Representation Models](https://mlsanity.com/paper/2303.07826/) (2023)
- [Call Me Maybe: Enhancing JavaScript Call Graph Construction using Graph Neural Networks](https://mlsanity.com/paper/2506.18191/) (2025)
- [Inferring Pluggable Types with Machine Learning](https://mlsanity.com/paper/2406.15676/) (2024)
- [TypyBench: Evaluating LLM Type Inference for Untyped Python Repositories](https://mlsanity.com/paper/2507.22086/) (2025)
- [Automated Type Annotation in Python Using Large Language Models](https://mlsanity.com/paper/2508.00422/) (2025)

## Abstract (arXiv)

Type inference over partial contexts in dynamically typed languages is challenging. In this work, we present a graph neural network model that predicts types by probabilistically reasoning over a program's structure, names, and patterns. The network uses deep similarity learning to learn a TypeSpace -- a continuous relaxation of the discrete space of types -- and how to embed the type properties of a symbol (i.e. identifier) into it. Importantly, our model can employ one-shot learning to predict an open vocabulary of types, including rare and user-defined ones. We realise our approach in Typilus for Python that combines the TypeSpace with an optional type checker. We show that Typilus accurately predicts types. Typilus confidently predicts types for 70% of all annotatable symbols; when it predicts a type, that type optionally type checks 95% of the time. Typilus can also find incorrect type annotations; two important and popular open source libraries, fairseq and allennlp, accepted our pull requests that fixed the annotation errors Typilus discovered.