# Falcon 7b for Software Mention Detection in Scholarly Documents

arXiv: https://arxiv.org/abs/2405.08514 · HTML: https://mlsanity.com/paper/2405.08514/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 14 May 2024.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [GPT-NER: Named Entity Recognition via Large Language Models](https://mlsanity.com/paper/2304.10428/) (2023)
- [SoMeSci- A 5 Star Open Data Gold Standard Knowledge Graph of Software Mentions in Scientific Articles](https://mlsanity.com/paper/2108.09070/) (2021)
- [Investigating Software Usage in the Social Sciences: A Knowledge Graph Approach](https://mlsanity.com/paper/2003.10715/) (2020)
- [E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition](https://mlsanity.com/paper/2305.17854/) (2023)
- [MexPub: Deep Transfer Learning for Metadata Extraction from German Publications](https://mlsanity.com/paper/2106.07359/) (2021)

## Abstract (arXiv)

This paper aims to tackle the challenge posed by the increasing integration of software tools in research across various disciplines by investigating the application of Falcon-7b for the detection and classification of software mentions within scholarly texts. Specifically, the study focuses on solving Subtask I of the Software Mention Detection in Scholarly Publications (SOMD), which entails identifying and categorizing software mentions from academic literature. Through comprehensive experimentation, the paper explores different training strategies, including a dual-classifier approach, adaptive sampling, and weighted loss scaling, to enhance detection accuracy while overcoming the complexities of class imbalance and the nuanced syntax of scholarly writing. The findings highlight the benefits of selective labelling and adaptive sampling in improving the model's performance. However, they also indicate that integrating multiple strategies does not necessarily result in cumulative improvements. This research offers insights into the effective application of large language models for specific tasks such as SOMD, underlining the importance of tailored approaches to address the unique challenges presented by academic text analysis.