# Cross-Lingual Transfer Learning for Statistical Type Inference

arXiv: https://arxiv.org/abs/2107.00157 · HTML: https://mlsanity.com/paper/2107.00157/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Exploiting Similarities among Languages for Machine Translation](https://mlsanity.com/paper/1309.4168/) (2013)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [A Convolutional Attention Network for Extreme Summarization of Source Code](https://mlsanity.com/paper/1602.03001/) (2016)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019)
- [Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1606.01614/) (2016)
- [LambdaNet: Probabilistic Type Inference using Graph Neural Networks](https://mlsanity.com/paper/2005.02161/) (2020)
- [TypeWriter: Neural Type Prediction with Search-based Validation](https://mlsanity.com/paper/1912.03768/) (2019)
- [Typilus: Neural Type Hints](https://mlsanity.com/paper/2004.10657/) (2020)
- [Robust Multilingual Part-of-Speech Tagging via Adversarial Training](https://mlsanity.com/paper/1711.04903/) (2017)
- [SPI: Automated Identification of Security Patches via Commits](https://mlsanity.com/paper/2105.14565/) (2021)
- [What's in a Domain? Learning Domain-Robust Text Representations using Adversarial Training](https://mlsanity.com/paper/1805.06088/) (2018)
- [SAR: Learning Cross-Language API Mappings with Little Knowledge](https://mlsanity.com/paper/1906.03835/) (2019)

## Built on by (most cited first, 2 of 2)

- [Callee: Recovering Call Graphs for Binaries with Transfer and Contrastive Learning](https://mlsanity.com/paper/2111.01415/) (2021)
- [An Aspect Extraction Framework using Different Embedding Types, Learning Models, and Dependency Structure](https://mlsanity.com/paper/2503.03512/) (2025)

## Abstract (arXiv)

Hitherto statistical type inference systems rely thoroughly on supervised learning approaches, which require laborious manual effort to collect and label large amounts of data. Most Turing-complete imperative languages share similar control- and data-flow structures, which make it possible to transfer knowledge learned from one language to another. In this paper, we propose a cross-lingual transfer learning framework, PLATO, for statistical type inference, which allows us to leverage prior knowledge learned from the labeled dataset of one language and transfer it to the others, e.g., Python to JavaScript, Java to JavaScript, etc. PLATO is powered by a novel kernelized attention mechanism to constrain the attention scope of the backbone Transformer model such that the model is forced to base its prediction on commonly shared features among languages. In addition, we propose the syntax enhancement that augments the learning on the feature overlap among language domains. Furthermore, PLATO can also be used to improve the performance of the conventional supervised learning-based type inference by introducing cross-lingual augmentation, which enables the model to learn more general features across multiple languages. We evaluated PLATO under two settings: 1) under the cross-domain scenario that the target language data is not labeled or labeled partially, the results show that PLATO outperforms the state-of-the-art domain transfer techniques by a large margin, e.g., it improves the Python to TypeScript baseline by +5.40%@EM, +5.40%@weighted-F1, and 2) under the conventional monolingual supervised learning based scenario, PLATO improves the Python baseline by +4.40%@EM, +3.20%@EM (parametric).