# TreeCaps: Tree-Based Capsule Networks for Source Code Processing

arXiv: https://arxiv.org/abs/2009.09777 · HTML: https://mlsanity.com/paper/2009.09777/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2020.

## Built on by (most cited first, 16 of 16)

- [HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale](https://mlsanity.com/paper/2409.16299/) (2024)
- [AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology](https://mlsanity.com/paper/2406.11912/) (2024)
- [The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation](https://mlsanity.com/paper/2305.06156/) (2023)
- [HierarchyNet: Learning to Summarize Source Code with Heterogeneous Representations](https://mlsanity.com/paper/2205.15479/) (2022)
- [CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking](https://mlsanity.com/paper/2402.11842/) (2024)
- [Multi-View Graph Representation for Programming Language Processing: An Investigation into Algorithm Detection](https://mlsanity.com/paper/2202.12481/) (2022)
- [Learning Program Semantics with Code Representations: An Empirical Study](https://mlsanity.com/paper/2203.11790/) (2022)
- [Learning Program Representations with a Tree-Structured Transformer](https://mlsanity.com/paper/2208.08643/) (2022)
- [TransformCode: A Contrastive Learning Framework for Code Embedding via Subtree Transformation](https://mlsanity.com/paper/2311.08157/) (2023)
- [Leveraging Artificial Intelligence on Binary Code Comprehension](https://mlsanity.com/paper/2210.05103/) (2022)
- [Energy-bounded Learning for Robust Models of Code](https://mlsanity.com/paper/2112.11226/) (2021)
- [Implant Global and Local Hierarchy Information to Sequence based Code Representation Models](https://mlsanity.com/paper/2303.07826/) (2023)
- [Heterogeneous Directed Hypergraph Neural Network over abstract syntax tree (AST) for Code Classification](https://mlsanity.com/paper/2305.04228/) (2023)
- [ParseCaps: An Interpretable Parsing Capsule Network for Medical Image Diagnosis](https://mlsanity.com/paper/2411.01564/) (2024)
- [TreeGPT: Pure TreeFFN Encoder-Decoder Architecture for Structured Reasoning Without Attention Mechanisms](https://mlsanity.com/paper/2509.05550/) (2025)
- [Test Case Recommendations with Distributed Representation of Code Syntactic Features](https://mlsanity.com/paper/2310.03174/) (2023)

## Abstract (arXiv)

Recently program learning techniques have been proposed to process source code based on syntactical structures (e.g., Abstract Syntax Trees) and/or semantic information (e.g., Dependency Graphs). Although graphs may be better at capturing various viewpoints of code semantics than trees, constructing graph inputs from code needs static code semantic analysis that may not be accurate and introduces noise during learning. Although syntax trees are precisely defined according to the language grammar and easier to construct and process than graphs, previous tree-based learning techniques have not been able to learn semantic information from trees to achieve better accuracy than graph-based techniques. We propose a new learning technique, named TreeCaps, by fusing together capsule networks with tree-based convolutional neural networks, to achieve learning accuracy higher than existing graph-based techniques while it is based only on trees. TreeCaps introduces novel variable-to-static routing algorithms into the capsule networks to compensate for the loss of previous routing algorithms. Aside from accuracy, we also find that TreeCaps is the most robust to withstand those semantic-preserving program transformations that change code syntax without modifying the semantics. Evaluated on a large number of Java and C/C++ programs, TreeCaps models outperform prior deep learning models of program source code, in terms of both accuracy and robustness for program comprehension tasks such as code functionality classification and function name prediction