# Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification

arXiv: https://arxiv.org/abs/2303.07142 · HTML: https://mlsanity.com/paper/2303.07142/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 13 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Paradigm Shift in Natural Language Processing](https://mlsanity.com/paper/2109.12575/) (2021)
- [The Unreasonable Effectiveness of the Baseline: Discussing SVMs in Legal Text Classification](https://mlsanity.com/paper/2109.07234/) (2021)

## Built on by (most cited first, 18 of 18)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Fake Alignment: Are LLMs Really Aligned Well?](https://mlsanity.com/paper/2311.05915/) (2023)
- [Exploring ChatGPT's Capabilities on Vulnerability Management](https://mlsanity.com/paper/2311.06530/) (2023)
- [VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation](https://mlsanity.com/paper/2403.12415/) (2024)
- [Fairness of ChatGPT and the Role Of Explainable-Guided Prompts](https://mlsanity.com/paper/2307.11761/) (2023)
- [Large Language Models for Propaganda Detection](https://mlsanity.com/paper/2310.06422/) (2023)
- [CogniDual Framework: Self-Training Large Language Models within a Dual-System Theoretical Framework for Improving Cognitive Tasks](https://mlsanity.com/paper/2409.03381/) (2024)
- [SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care](https://mlsanity.com/paper/2601.16529/) (2026)
- [Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Framework](https://mlsanity.com/paper/2409.04744/) (2024)
- [SpamDam: Towards Privacy-Preserving and Adversary-Resistant SMS Spam Detection](https://mlsanity.com/paper/2404.09481/) (2024)
- [Can GPT-4o mini and Gemini 2.0 Flash Predict Fine-Grained Fashion Product Attributes? A Zero-Shot Analysis](https://mlsanity.com/paper/2507.09950/) (2025)
- [Evaluating Large Language Models for Material Selection](https://mlsanity.com/paper/2405.03695/) (2024)
- [Confounder Balancing in Adversarial Domain Adaptation for Pre-Trained Large Models Fine-Tuning](https://mlsanity.com/paper/2310.16062/) (2023)
- [Neurosymbolic Inference On Foundation Models For Remote Sensing Text-to-image Retrieval With Complex Queries](https://mlsanity.com/paper/2512.14102/) (2025)
- [Towards Fairer Health Recommendations: finding informative unbiased samples via Word Sense Disambiguation](https://mlsanity.com/paper/2409.07424/) (2024)
- [LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents](https://mlsanity.com/paper/2509.02241/) (2025)
- [Evaluating the Effects of Prompt Perturbation on Bias and Hallucination in Large Language Models](https://mlsanity.com/paper/2609.35804/) (2026)

## Abstract (arXiv)

This case study investigates the task of job classification in a real-world setting, where the goal is to determine whether an English-language job posting is appropriate for a graduate or entry-level position. We explore multiple approaches to text classification, including supervised approaches such as traditional models like Support Vector Machines (SVMs) and state-of-the-art deep learning methods such as DeBERTa. We compare them with Large Language Models (LLMs) used in both few-shot and zero-shot classification settings. To accomplish this task, we employ prompt engineering, a technique that involves designing prompts to guide the LLMs towards the desired output. Specifically, we evaluate the performance of two commercially available state-of-the-art GPT-3.5-based language models, text-davinci-003 and gpt-3.5-turbo. We also conduct a detailed analysis of the impact of different aspects of prompt engineering on the model's performance. Our results show that, with a well-designed prompt, a zero-shot gpt-3.5-turbo classifier outperforms all other models, achieving a 6% increase in Precision@95% Recall compared to the best supervised approach. Furthermore, we observe that the wording of the prompt is a critical factor in eliciting the appropriate "reasoning" in the model, and that seemingly minor aspects of the prompt significantly affect the model's performance.