# Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies arXiv: https://arxiv.org/abs/2310.18729 ยท HTML: https://mlsanity.com/paper/2310.18729/ ## Key facts - Cited by 25 papers in the map (data through 14 Oct 2026). - Builds on 5 of its 25 references that are in the map. - First posted on arXiv 28 Oct 2023. ## Builds on (most cited first, 5 of 5) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts](https://mlsanity.com/paper/2305.04417/) (2023) - [Can Large Language Models emulate an inductive Thematic Analysis of semi-structured interviews? An exploration and provocation on the limits of the approach and the model](https://mlsanity.com/paper/2305.13014/) (2023) - [Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domain](https://mlsanity.com/paper/2112.05807/) (2021) - [Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents](https://mlsanity.com/paper/2112.11494/) (2021) ## Built on by (most cited first, 25 of 25) - [Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey](https://mlsanity.com/paper/2503.15850/) (2025) - [Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis](https://mlsanity.com/paper/2406.10794/) (2024) - [Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models](https://mlsanity.com/paper/2502.06419/) (2025) - [Robots in the Middle: Evaluating LLMs in Dispute Resolution](https://mlsanity.com/paper/2410.07053/) (2024) - [Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond](https://mlsanity.com/paper/2503.16040/) (2025) - [TAMA: A Human-AI Collaborative Thematic Analysis Framework Using Multi-Agent LLMs for Clinical Interviews](https://mlsanity.com/paper/2503.20666/) (2025) - [Reflections on Inductive Thematic Saturation as a potential metric for measuring the validity of an inductive Thematic Analysis with LLMs](https://mlsanity.com/paper/2401.03239/) (2024) - [It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human](https://mlsanity.com/paper/2407.06798/) (2024) - [Using LLMs to Discover Legal Factors](https://mlsanity.com/paper/2410.07504/) (2024) - [Analyzing Nobel Prize Literature with Large Language Models](https://mlsanity.com/paper/2410.18142/) (2024) - [Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justice](https://mlsanity.com/paper/2412.15260/) (2024) - [Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgment](https://mlsanity.com/paper/2405.12910/) (2024) - [Codebook Reduction and Saturation: Novel observations on Inductive Thematic Saturation for Large Language Models and initial coding in Thematic Analysis](https://mlsanity.com/paper/2503.04859/) (2025) - [Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification](https://mlsanity.com/paper/2507.05010/) (2025) - [SFT-TA: Supervised Fine-Tuned Agents in Multi-Agent LLMs for Automated Inductive Thematic Analysis](https://mlsanity.com/paper/2509.17167/) (2025) - [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024) - [LLM-TA: An LLM-Enhanced Thematic Analysis Pipeline for Transcripts from Parents of Children with Congenital Heart Disease](https://mlsanity.com/paper/2502.01620/) (2025) - [Human vs. LLM-Based Thematic Analysis for Digital Mental Health Research: Proof-of-Concept Comparative Study](https://mlsanity.com/paper/2507.08002/) (2025) - [NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models](https://mlsanity.com/paper/2606.15646/) (2026) - [Impacts of Continued Legal Pre-Training and IFT on LLMs' Latent Representations of Human-Defined Legal Concepts](https://mlsanity.com/paper/2410.12001/) (2024) - [VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents](https://mlsanity.com/paper/2506.21582/) (2025) - [Retrieving Semantically Similar Decisions under Noisy Institutional Labels: Robust Comparison of Embedding Methods](https://mlsanity.com/paper/2512.05681/) (2025) - [Beyond the Star Rating: A Scalable Framework for Aspect-Based Sentiment Analysis Using LLMs and Text Classification](https://mlsanity.com/paper/2602.21082/) (2026) - [LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metrics](https://mlsanity.com/paper/2605.13412/) (2026) - [In With the Old: Enhancing 'Classical' Document Automation with Generative AI](https://mlsanity.com/paper/2610.07480/) (2026) ## Abstract (arXiv) Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4). We employed the framework for an analysis of a dataset (n=785) of facts descriptions from criminal court opinions regarding thefts. The goal of the analysis was to discover classes of typical thefts. Our results show that the LLM, namely OpenAI's GPT-4, generated reasonable initial codes, and it was capable of improving the quality of the codes based on expert feedback. They also suggest that the model performed well in zero-shot classification of facts descriptions in terms of the themes. Finally, the themes autonomously discovered by the LLM appear to map fairly well to the themes arrived at by legal experts. These findings can be leveraged by legal researchers to guide their decisions in integrating LLMs into their thematic analyses, as well as other inductive coding projects.