mlsanityData to 14 Oct 2026

Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies

arXiv 2310.18729 ↗First posted 28 Oct 2023cs.AI cs.CL cs.HCcited by 25

Not on the map. Why?

Abstract (arXiv)

Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4). We employed the framework for an analysis of a dataset (n=785) of facts descriptions from criminal court opinions regarding thefts. The goal of the analysis was to discover classes of typical thefts. Our results show that the LLM, namely OpenAI's GPT-4, generated reasonable initial codes, and it was capable of improving the quality of the codes based on expert feedback. They also suggest that the model performed well in zero-shot classification of facts descriptions in terms of the themes. Finally, the themes autonomously discovered by the LLM appear to map fairly well to the themes arrived at by legal experts. These findings can be leveraged by legal researchers to guide their decisions in integrating LLMs into their thematic analyses, as well as other inductive coding projects.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 5 of its 25 references that are in the map. First posted on arXiv 28 Oct 2023.

Builds on

5 of 25 references.

References · 5

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Textscited by 33
  3. 2023Can Large Language Models emulate an inductive Thematic Analysis of semi-structured interviews? An exploration and provocation on the limits of the approach and the modelcited by 15
  4. 2021Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domaincited by 12
  5. 2021Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documentscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 11; 2026: 4.

25 papers.

Sort:

By area:

  1. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 95
  2. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 64
  3. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 28
  4. 2024Robots in the Middle: Evaluating LLMs in Dispute Resolutioncited by 14
  5. 2025Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyondcited by 9
  6. 2025TAMA: A Human-AI Collaborative Thematic Analysis Framework Using Multi-Agent LLMs for Clinical Interviewscited by 9
  7. 2024Reflections on Inductive Thematic Saturation as a potential metric for measuring the validity of an inductive Thematic Analysis with LLMscited by 6
  8. 2024It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Humancited by 5
  9. 2024Using LLMs to Discover Legal Factorscited by 5
  10. 2024Analyzing Nobel Prize Literature with Large Language Modelscited by 4
  11. 2024Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justicecited by 4
  12. 2024Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgmentcited by 3
13 more
  1. 2025Codebook Reduction and Saturation: Novel observations on Inductive Thematic Saturation for Large Language Models and initial coding in Thematic Analysiscited by 3
  2. 2025Co-DETECT: Collaborative Discovery of Edge Cases in Text Classificationcited by 3
  3. 2025SFT-TA: Supervised Fine-Tuned Agents in Multi-Agent LLMs for Automated Inductive Thematic Analysiscited by 3
  4. 2024Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysiscited by 2
  5. 2025LLM-TA: An LLM-Enhanced Thematic Analysis Pipeline for Transcripts from Parents of Children with Congenital Heart Diseasecited by 1
  6. 2025Human vs. LLM-Based Thematic Analysis for Digital Mental Health Research: Proof-of-Concept Comparative Studycited by 1
  7. 2026NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Modelscited by 1
  8. 2024Impacts of Continued Legal Pre-Training and IFT on LLMs' Latent Representations of Human-Defined Legal Conceptscited by 0
  9. 2025VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agentscited by 0
  10. 2025Retrieving Semantically Similar Decisions under Noisy Institutional Labels: Robust Comparison of Embedding Methodscited by 0
  11. 2026Beyond the Star Rating: A Scalable Framework for Aspect-Based Sentiment Analysis Using LLMs and Text Classificationcited by 0
  12. 2026LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metricscited by 0
  13. 2026In With the Old: Enhancing 'Classical' Document Automation with Generative AIcited by 0