# TaskComplexity: A Dataset for Task Complexity Classification with In-Context Learning, FLAN-T5 and GPT-4o Benchmarks

arXiv: https://arxiv.org/abs/2409.20189 · HTML: https://mlsanity.com/paper/2409.20189/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2404.05231/) (2024)
- [Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family](https://mlsanity.com/paper/2303.07992/) (2023)
- [Fairness-guided Few-shot Prompting for Large Language Models](https://mlsanity.com/paper/2303.13217/) (2023)
- [Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly](https://mlsanity.com/paper/2310.03150/) (2023)
- [Efficiency optimization of large-scale language models based on deep learning in natural language processing tasks](https://mlsanity.com/paper/2405.11704/) (2024)
- [HaT5: Hate Language Identification using Text-to-Text Transfer Transformer](https://mlsanity.com/paper/2202.05690/) (2022)
- [NLP-LTU at SemEval-2023 Task 10: The Impact of Data Augmentation and Semi-Supervised Learning Techniques on Text Classification Performance on an Imbalanced Dataset](https://mlsanity.com/paper/2304.12847/) (2023)
- [Mashee at SemEval-2024 Task 8: The Impact of Samples Quality on the Performance of In-Context Learning for Machine Text Classification](https://mlsanity.com/paper/2406.17790/) (2024)

## Abstract (arXiv)

This paper addresses the challenge of classifying and assigning programming tasks to experts, a process that typically requires significant effort, time, and cost. To tackle this issue, a novel dataset containing a total of 4,112 programming tasks was created by extracting tasks from various websites. Web scraping techniques were employed to collect this dataset of programming problems systematically. Specific HTML tags were tracked to extract key elements of each issue, including the title, problem description, input-output, examples, problem class, and complexity score. Examples from the dataset are provided in the appendix to illustrate the variety and complexity of tasks included. The dataset's effectiveness has been evaluated and benchmarked using two approaches; the first approach involved fine-tuning the FLAN-T5 small model on the dataset, while the second approach used in-context learning (ICL) with the GPT-4o mini. The performance was assessed using standard metrics: accuracy, recall, precision, and F1-score. The results indicated that in-context learning with GPT-4o-mini outperformed the FLAN-T5 model.