mlsanityData to 7 Oct 2026

(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice

arXiv 2402.01864 ↗First posted 2 Feb 2024cs.CY cs.AIcited by 61

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly capable of providing users with advice in a wide range of professional domains, including legal advice. However, relying on LLMs for legal queries raises concerns due to the significant expertise required and the potential real-world consequences of the advice. To explore textitwhen and textitwhy LLMs should or should not provide advice to users, we conducted workshops with 20 legal experts using methods inspired by case-based reasoning. The provided realistic queries ("cases") allowed experts to examine granular, situation-specific concerns and overarching technical and legal constraints, producing a concrete set of contextual considerations for LLM developers. By synthesizing the factors that impacted LLM response appropriateness, we present a 4-dimension framework: (1) User attributes and behaviors, (2) Nature of queries, (3) AI capabilities, and (4) Social impacts. We share experts' recommendations for LLM response strategies, which center around helping users identify `right questions to ask' and relevant information rather than providing definitive legal judgments. Our findings reveal novel legal considerations, such as unauthorized practice of law, confidentiality, and liability for inaccurate advice, that have been overlooked in the literature. The case-based deliberation method enabled us to elicit fine-grained, practice-informed insights that surpass those from de-contextualized surveys or speculative principles. These findings underscore the applicability of our method for translating domain-specific professional knowledge and practices into policies that can guide LLM behavior in a more responsible direction.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 36 of its 101 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

36 of 101 references.

References · 36

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  4. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  5. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  6. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  7. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  8. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  9. 2023Explainability for Large Language Models: A Surveycited by 331
  10. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328
  11. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  12. 2023Role-Play with Large Language Modelscited by 290
24 more
  1. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 199
  2. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  3. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  4. 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 137
  5. 2021Lawformer: A Pre-trained Language Model for Chinese Legal Long Documentscited by 115
  6. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  7. 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practicecited by 86
  8. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  9. 2022Legal Prompt Engineering for Multilingual Legal Judgement Predictioncited by 54
  10. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  11. 2023Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Predictioncited by 37
  12. 2022Augmenting Interpretable Models with LLMs during Trainingcited by 36
  13. 2023Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Modelscited by 35
  14. 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 33
  15. 2023ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languagescited by 33
  16. 2023Understanding Users' Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Levelcited by 26
  17. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  18. 2022On the Role of Negative Precedent in Legal Outcome Predictioncited by 16
  19. 2023Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Applicationcited by 14
  20. 2024Promises and pitfalls of artificial intelligence for legal applicationscited by 14
  21. 2023Large Language Models as Corporate Lobbyistscited by 13
  22. 2023Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Modelscited by 13
  23. 2023Bringing order into the realm of Transformer-based language models for artificial intelligence and lawcited by 13
  24. 2023Enhancing Logical Reasoning in Large Language Models to Facilitate Legal Applicationscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 28; 2026: 16.

61 papers.

Sort:

By area:

  1. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  2. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  3. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  4. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  5. 2024How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensionscited by 51
  6. 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
  7. 2024LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Modelscited by 36
  8. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  9. 2025Data Poisoning in Deep Learning: A Surveycited by 20
  10. 2024From Text to Transformation: A Comprehensive Review of Large Language Models' Versatilitycited by 19
  11. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  12. 2024Uncertainty Quantification in Large Language Models Through Convex Hull Analysiscited by 12
38 more
  1. 2024The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Makingcited by 12
  2. 2025What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Textcited by 9
  3. 2025Too Helpful, Too Harmless, Too Honest or Just Right?cited by 9
  4. 2024'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMscited by 7
  5. 2024SPICA: Retrieving Scenarios for Pluralistic In-Context Alignmentcited by 6
  6. 2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
  7. 2024It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Humancited by 5
  8. 2024CUIfy the XR: An Open-Source Package to Embed LLM-powered Conversational Agents in XRcited by 5
  9. 2024Large Language Models in Politics and Democracy: A Comprehensive Surveycited by 5
  10. 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
  11. 2026Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoderscited by 5
  12. 2024Responsible Retrieval Augmented Generation for Climate Decision Making from Documentscited by 4
  13. 2025ThinkEval: Practical Evaluation of Knowledge Leakage in LLM Editing using Thought-based Knowledge Graphscited by 4
  14. 2025ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanationscited by 4
  15. 2026Adaptive Collaboration of Arena-Based Argumentative LLMs for Explainable and Contestable Legal Reasoningcited by 4
  16. 2024Pre-Trained Language Models for Keyphrase Prediction: A Reviewcited by 3
  17. 2025Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?cited by 3
  18. 2025MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Languagecited by 3
  19. 2025GENUINE: Graph Enhanced Multi-level Uncertainty Estimation for Large Language Modelscited by 3
  20. 2025Statistical Runtime Verification for LLMs via Robustness Estimationcited by 2
  21. 2025AgriDoctor: A Multimodal Intelligent Assistant for Agriculturecited by 2
  22. 2025Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metricscited by 2
  23. 2025The Epistemological Consequences of Large Language Models: Rethinking collective intelligence and institutional knowledgecited by 2
  24. 2026An Empirical Study of Collective Behaviors and Social Dynamics in Large Language Model Agentscited by 2
  25. 2024Output Scouting: Auditing Large Language Models for Catastrophic Responsescited by 1
  26. 2025Multi-Stage Retrieval for Operational Technology Cybersecurity Compliance Using Large Language Models: A Railway Casestudycited by 1
  27. 2025A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standardscited by 1
  28. 2025Can Large Language Models Capture Human Risk Preferences? A Cross-Cultural Studycited by 1
  29. 2025'Rich Dad, Poor Lad': How do Large Language Models Contextualize Socioeconomic Factors in College Admission ?cited by 1
  30. 2026Growth First, Care Second? Tracing the Landscape of LLM Value Preferences in Everyday Dilemmascited by 1
  31. 2026When the Model Said 'No Comment', We Knew Helpfulness Was Dead, Honesty Was Alive, and Safety Was Terrifiedcited by 1
  32. 2026Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshapingcited by 1
  33. 2025Lifelong Reinforcement Learning with Similarity-Driven Weighting by Large Modelscited by 0
  34. 2025GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabricationcited by 0
  35. 2025FedSODA: Federated Fine-tuning of LLMs via Similarity Group Pruning and Orchestrated Distillation Alignmentcited by 0
  36. 2025PolicyPad: Collaborative Prototyping of LLM Policiescited by 0
  37. 2025Towards Human-Centered RegTech: Unpacking Professionals' Strategies and Needs for Using LLMs Safelycited by 0
  38. 2025OptAgent: Optimizing Query Rewriting for E-commerce via Multi-Agent Simulationcited by 0

50 most cited of 61 All 61 (JSON)