# Instruction Tuning Vs. In-Context Learning: Revisiting Large Language Models in Few-Shot Computational Social Science

arXiv: https://arxiv.org/abs/2409.14673 · HTML: https://mlsanity.com/paper/2409.14673/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation](https://mlsanity.com/paper/2305.16938/) (2023)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification](https://mlsanity.com/paper/2406.08660/) (2024)
- [The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities](https://mlsanity.com/paper/2408.13296/) (2024)
- [Probing LLMs for hate speech detection: strengths and vulnerabilities](https://mlsanity.com/paper/2310.12860/) (2023)
- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Automatically Identifying Complaints in Social Media](https://mlsanity.com/paper/1906.03890/) (2019)
- [Fine-tuning Large Language Models with Sequential Instructions](https://mlsanity.com/paper/2403.07794/) (2024)
- [Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science](https://mlsanity.com/paper/2305.14310/) (2023)
- [Exploring the Relationship between In-Context Learning and Instruction Tuning](https://mlsanity.com/paper/2311.10367/) (2023)
- [Automatic Identification and Classification of Bragging in Social Media](https://mlsanity.com/paper/2203.05840/) (2022)
- [Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning](https://mlsanity.com/paper/2310.08309/) (2023)
- [Team QUST at SemEval-2023 Task 3: A Comprehensive Study of Monolingual and Multilingual Approaches for Detecting Online News Genre, Framing and Persuasion Techniques](https://mlsanity.com/paper/2304.04190/) (2023)
- [Prompt Refinement or Fine-tuning? Best Practices for using LLMs in Computational Social Science Tasks](https://mlsanity.com/paper/2408.01346/) (2024)
- [Team QUST at SemEval-2024 Task 8: A Comprehensive Study of Monolingual and Multilingual Approaches for Detecting AI-generated Text](https://mlsanity.com/paper/2402.11934/) (2024)

## Built on by (most cited first, 5 of 5)

- [Generative AI collective behavior needs an interactionist paradigm](https://mlsanity.com/paper/2601.10567/) (2026)
- [AMPLE: Emotion-Aware Multimodal Fusion Prompt Learning for Fake News Detection](https://mlsanity.com/paper/2410.15591/) (2024)
- [The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2512.22337/) (2025)
- [Uncovering inequalities in new knowledge learning by large language models across different languages](https://mlsanity.com/paper/2503.04064/) (2025)
- [Magic Words or Methodical Work? Challenging Conventional Wisdom in LLM-Based Political Text Annotation](https://mlsanity.com/paper/2603.26898/) (2026)

## Abstract (arXiv)

Real-world applications of large language models (LLMs) in computational social science (CSS) tasks primarily depend on the effectiveness of instruction tuning (IT) or in-context learning (ICL). While IT has shown highly effective at fine-tuning LLMs for various tasks, ICL offers a rapid alternative for task adaptation by learning from examples without explicit gradient updates. In this paper, we evaluate the classification performance of LLMs using IT versus ICL in few-shot CSS tasks. The experimental results indicate that ICL consistently outperforms IT in most CSS tasks. Additionally, we investigate the relationship between the increasing number of training samples and LLM performance. Our findings show that simply increasing the number of samples without considering their quality does not consistently enhance the performance of LLMs with either ICL or IT and can sometimes even result in a performance decline. Finally, we compare three prompting strategies, demonstrating that ICL is more effective than zero-shot and Chain-of-Thought (CoT). Our research highlights the significant advantages of ICL in handling CSS tasks in few-shot settings and emphasizes the importance of optimizing sample quality and prompting strategies to improve LLM classification performance. The code will be made available.