# NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classification

arXiv: https://arxiv.org/abs/2402.16910 · HTML: https://mlsanity.com/paper/2402.16910/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 25 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Neurosymbolic AI: The 3rd Wave](https://mlsanity.com/paper/2012.05876/) (2020)
- [ADaPT: As-Needed Decomposition and Planning with Language Models](https://mlsanity.com/paper/2311.05772/) (2023)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Compositional Semantic Parsing with Large Language Models](https://mlsanity.com/paper/2209.15003/) (2022)
- [A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends](https://mlsanity.com/paper/2311.10372/) (2023)
- [Comprehensive Exploration of Synthetic Data Generation: A Survey](https://mlsanity.com/paper/2401.02524/) (2024)
- [SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models](https://mlsanity.com/paper/2205.07381/) (2022)
- [Is a Question Decomposition Unit All We Need?](https://mlsanity.com/paper/2205.12538/) (2022)
- [Natural Language Decomposition and Interpretation of Complex Utterances](https://mlsanity.com/paper/2305.08677/) (2023)
- [Does Synthetic Data Make Large Language Models More Efficient?](https://mlsanity.com/paper/2310.07830/) (2023)
- ["Understanding AI": Semantic Grounding in Large Language Models](https://mlsanity.com/paper/2402.10992/) (2024)
- [ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models](https://mlsanity.com/paper/2212.10815/) (2022)
- [NeSIG: A Neuro-Symbolic Method for Learning to Generate Planning Problems](https://mlsanity.com/paper/2301.10280/) (2023)
- [Semantic Composition and Decomposition: From Recognition to Generation](https://mlsanity.com/paper/1405.7908/) (2014)
- [Distilling LLMs' Decomposition Abilities into Compact Language Models](https://mlsanity.com/paper/2402.01812/) (2024)
- [Mitigating Data Scarcity for Large Language Models](https://mlsanity.com/paper/2302.01806/) (2023)
- [A ML-LLM pairing for better code comment classification](https://mlsanity.com/paper/2310.10275/) (2023)
- [Semantic Decomposition Improves Learning of Large Language Models on EHR Data](https://mlsanity.com/paper/2212.06040/) (2022)
- [Generative AI for Software Metadata: Overview of the Information Retrieval in Software Engineering Track at FIRE 2023](https://mlsanity.com/paper/2311.03374/) (2023)

## Abstract (arXiv)

We present a neuro-symbolic (NeSy) workflow combining a symbolic-based learning technique with a large language model (LLM) agent to generate synthetic data for code comment classification in the C programming language. We also show how generating controlled synthetic data using this workflow fixes some of the notable weaknesses of LLM-based generation and increases the performance of classical machine learning models on the code comment classification task. Our best model, a Neural Network, achieves a Macro-F1 score of 91.412% with an increase of 1.033% after data augmentation.