# MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning

arXiv: https://arxiv.org/abs/2206.06460 · HTML: https://mlsanity.com/paper/2206.06460/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 13 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [Rethinking Positional Encoding in Language Pre-training](https://mlsanity.com/paper/2006.15595/) (2020)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2103.12579/) (2021)
- [Learning feed-forward one-shot learners](https://mlsanity.com/paper/1606.05233/) (2016)
- [A Convolutional Attention Network for Extreme Summarization of Source Code](https://mlsanity.com/paper/1602.03001/) (2016)
- [Code Prediction by Feeding Trees to Transformers](https://mlsanity.com/paper/2003.13848/) (2020)
- [DeepBugs: A Learning Approach to Name-based Bug Detection](https://mlsanity.com/paper/1805.11683/) (2018)
- [Language-Agnostic Representation Learning of Source Code from Structure and Context](https://mlsanity.com/paper/2103.11318/) (2021)
- [Multi-task Learning based Pre-trained Language Model for Code Completion](https://mlsanity.com/paper/2012.14631/) (2020)
- [Structured Neural Summarization](https://mlsanity.com/paper/1811.01824/) (2018)
- [TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning](https://mlsanity.com/paper/1904.05967/) (2019)
- [Meta Multi-Task Learning for Sequence Modeling](https://mlsanity.com/paper/1802.08969/) (2018)
- [A Self-Attentional Neural Architecture for Code Completion with Multi-Task Learning](https://mlsanity.com/paper/1909.06983/) (2019)

## Built on by (most cited first, 6 of 6)

- [IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators](https://mlsanity.com/paper/2403.03894/) (2024)
- [Just-in-Time Detection of Silent Security Patches](https://mlsanity.com/paper/2312.01241/) (2023)
- [Cross-lingual Transfer in Programming Languages: An Extensive Empirical Study](https://mlsanity.com/paper/2310.16937/) (2023)
- [Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents](https://mlsanity.com/paper/2508.04412/) (2025)
- [Zero-Shot Cross-Domain Code Search without Fine-Tuning](https://mlsanity.com/paper/2504.07740/) (2025)
- [LaFiCMIL: Rethinking Large File Classification from the Perspective of Correlated Multiple Instance Learning](https://mlsanity.com/paper/2308.01413/) (2023)

## Abstract (arXiv)

Representation learning of source code is essential for applying machine learning to software engineering tasks. Learning code representation from a multilingual source code dataset has been shown to be more effective than learning from single-language datasets separately, since more training data from multilingual dataset improves the model's ability to extract language-agnostic information from source code. However, existing multilingual training overlooks the language-specific information which is crucial for modeling source code across different programming languages, while only focusing on learning a unified model with shared parameters among different languages for language-agnostic information modeling. To address this problem, we propose MetaTPTrans, a meta learning approach for multilingual code representation learning. MetaTPTrans generates different parameters for the feature extractor according to the specific programming language type of the input code snippet, enabling the model to learn both language-agnostic and language-specific information with dynamic parameters in the feature extractor. We conduct experiments on the code summarization and code completion tasks to verify the effectiveness of our approach. The results demonstrate the superiority of our approach with significant improvements on state-of-the-art baselines.