Na Pr'atica, qual IA Entende o Direito? Um Estudo Experimental com IAs Generalistas e uma IA Jur'idica
Not on the map. Why?
Abstract (arXiv)
This study presents the Jusbrasil Study on the Use of General-Purpose AIs in Law, proposing an experimental evaluation protocol combining legal theory, such as material correctness, systematic coherence, and argumentative integrity, with empirical assessment by 48 legal professionals. Four systems (JusIA, ChatGPT Free, ChatGPT Pro, and Gemini) were tested in tasks simulating lawyers' daily work. JusIA, a domain-specialized model, consistently outperformed the general-purpose systems, showing that both domain specialization and a theoretically grounded evaluation are essential for reliable legal AI outputs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 25 references that are in the map. First posted on arXiv 20 Oct 2025.
Builds on
9 of 25 references.
References · 9
- 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
- 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
- 2024LLM-based NLG Evaluation: Current Status and Challengescited by 73
- 2025JuDGE: Benchmarking Judgment Document Generation for Chinese Legal Systemcited by 18
- 2025Automatic Legal Writing Evaluation of LLMscited by 12
- 2022An Evaluation Framework for Legal Document Summarizationcited by 10
- 2025CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generationcited by 8
- 2023Question-Answering Approach to Evaluating Legal Summariescited by 5
- 2025LRAGE: Legal Retrieval Augmented Generation Evaluation Toolcited by 5
Built on by
Not cited yet.