mlsanityData to 14 Oct 2026

LawFlow: Collecting and Simulating Lawyers' Thought Processes on Business Formation Case Studies

arXiv 2504.18942 ↗First posted 26 Apr 2025cs.CL cs.AI cs.LGcited by 2

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Legal practitioners, particularly those early in their careers, face complex, high-stakes tasks that require adaptive, context-sensitive reasoning. While AI holds promise in supporting legal work, current datasets and models are narrowly focused on isolated subtasks and fail to capture the end-to-end decision-making required in real-world practice. To address this gap, we introduce LawFlow, a dataset of complete end-to-end legal workflows collected from trained law students, grounded in real-world business entity formation scenarios. Unlike prior datasets focused on input-output pairs or linear chains of thought, LawFlow captures dynamic, modular, and iterative reasoning processes that reflect the ambiguity, revision, and client-adaptive strategies of legal practice. Using LawFlow, we compare human and LLM-generated workflows, revealing systematic differences in structure, reasoning flexibility, and plan execution. Human workflows tend to be modular and adaptive, while LLM workflows are more sequential, exhaustive, and less sensitive to downstream implications. Our findings also suggest that legal professionals prefer AI to carry out supportive roles, such as brainstorming, identifying blind spots, and surfacing alternatives, rather than executing complex workflows end-to-end. Our results highlight both the current limitations of LLMs in supporting complex legal workflows and opportunities for developing more collaborative, reasoning-aware legal AI systems. All data and code are available on our project page (https://minnesotanlp.github.io/LawFlow-website/).

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 17 of its 34 references that are in the map. First posted on arXiv 26 Apr 2025.

Builds on

17 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024OpenAI o1 System Card21 Dec

Other references · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  4. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  5. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
  6. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Modelscited by 94
  7. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  8. 2024LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Modelscited by 37
  9. 2024WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Modelscited by 30
  10. 2023ProAgent: From Robotic Process Automation to Agentic Process Automationcited by 26
  11. 2020Robotic Process Automation -- A Systematic Literature Review and Assessment Frameworkcited by 9
  12. 2023LexGPT 0.1: pre-trained GPT-J models with Pile of Lawcited by 9
4 more
  1. 2025ScholaWrite: A Dataset of End-to-End Scholarly Writing Processcited by 9
  2. 2020On the Evaluation of Intelligent Process Automationcited by 6
  3. 2023Decoding the End-to-end Writing Trajectory in Scholarly Manuscriptscited by 3
  4. 2025AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Predictioncited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
  2. 2024A Comprehensive Framework for Reliable Legal AI: Combining Specialized Expert Systems and Adaptive Refinement
  3. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence
  4. 2025Benchmarking Multi-Step Legal Reasoning and Analyzing Chain-of-Thought Effects in Large Language Models
  5. 2025Thinking Longer, Not Always Smarter: Evaluating LLM Capabilities in Hierarchical Legal Reasoning
  6. 2026BenGER: Benchmarking LLM Systems on Subsumption-Based Legal Reasoning in German Law
  7. 2025Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession
  8. 2025Continual Pre-Training is (not) What You Need in Domain Adaption