# Generative AI for Software Metadata: Overview of the Information Retrieval in Software Engineering Track at FIRE 2023

arXiv: https://arxiv.org/abs/2311.03374 · HTML: https://mlsanity.com/paper/2311.03374/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 14 references that are in the map.
- First posted on arXiv 27 Oct 2023.

## Builds on (most cited first, 2 of 2)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Predicting Usefulness of Code Review Comments using Textual Features and Developer Experience](https://mlsanity.com/paper/1807.04485/) (2018)

## Built on by (most cited first, 1 of 1)

- [NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classification](https://mlsanity.com/paper/2402.16910/) (2024)

## Abstract (arXiv)

The Information Retrieval in Software Engineering (IRSE) track aims to develop solutions for automated evaluation of code comments in a machine learning framework based on human and large language model generated labels. In this track, there is a binary classification task to classify comments as useful and not useful. The dataset consists of 9048 code comments and surrounding code snippet pairs extracted from open source github C based projects and an additional dataset generated individually by teams using large language models. Overall 56 experiments have been submitted by 17 teams from various universities and software companies. The submissions have been evaluated quantitatively using the F1-Score and qualitatively based on the type of features developed, the supervised learning model used and their corresponding hyper-parameters. The labels generated from large language models increase the bias in the prediction model but lead to less over-fitted results.