mlsanityData to 14 Oct 2026

MLCPD: A Unified Multi-Language Code Parsing Dataset with Universal AST Schema

arXiv 2510.16357 ↗First posted 18 Oct 2025cs.SE cs.LG cs.PLcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce the MultiLang Code Parser Dataset (MLCPD), a large-scale, language-agnostic dataset unifying syntactic and structural representations of code across ten major programming languages. MLCPD contains over seven million parsed source files normalized under our proposed universal Abstract Syntax Tree (AST) schema, enabling consistent cross-language reasoning, structural learning, and multilingual software analysis. Unlike existing corpora that focus purely on token-level code or isolated parsers, MLCPD provides both hierarchical tree representations and rich metadata for every file, ensuring lossless syntactic coverage and structural uniformity. Each entry includes a normalized schema, language-level metadata, and abstracted node semantics stored in Parquet format for scalable retrieval. Empirical analyses reveal strong cross-language structural regularities-demonstrating that syntactic graphs from languages as diverse as Python, Java, and Go can be aligned under a shared schema. We release the dataset publicly on Hugging Face and the accompanying codebase on GitHub, which includes complete pipelines for dataset reproduction, grammar compilation, and a visualization tool for exploring the unified AST across languages. Together, these resources establish MLCPD as an open, reproducible foundation for future research in cross-language representation learning and program analysis.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 10 of its 32 references that are in the map. First posted on arXiv 18 Oct 2025.

Builds on

10 of 32 references.

References · 10

  1. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  2. 2023StarCoder: may the source be with you!cited by 799
  3. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  4. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  5. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  6. 2024AST-T5: Structure-Aware Pretraining for Code Generation and Understandingcited by 33
  7. 2023Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMscited by 19
  8. 2023VGX: Large-Scale Sample Generation for Boosting Learning-Based Software Vulnerability Analysescited by 9
  9. 2024A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Taskscited by 5
  10. 2019Static Code Analysis of Multilanguage Software Systemscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.