# CoDefeater: Using LLMs To Find Defeaters in Assurance Cases

arXiv: https://arxiv.org/abs/2407.13717 · HTML: https://mlsanity.com/paper/2407.13717/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 71 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning](https://mlsanity.com/paper/2401.06805/) (2024)
- [Few-shot training LLMs for project-specific code-summarization](https://mlsanity.com/paper/2207.04237/) (2022)
- [LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis](https://mlsanity.com/paper/2310.15100/) (2023)
- [Prompt-Enhanced Software Vulnerability Detection Using ChatGPT](https://mlsanity.com/paper/2308.12697/) (2023)
- [Large Language Model for Vulnerability Detection: Emerging Results and Future Directions](https://mlsanity.com/paper/2401.15468/) (2024)
- [Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs](https://mlsanity.com/paper/2310.13976/) (2023)
- [Can Large Language Models assist in Hazard Analysis?](https://mlsanity.com/paper/2303.15473/) (2023)
- [Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases](https://mlsanity.com/paper/2401.17991/) (2024)
- [Safety Case Templates for Autonomous Systems](https://mlsanity.com/paper/2102.02625/) (2021)
- [A PRISMA-driven systematic mapping study on system assurance weakeners](https://mlsanity.com/paper/2311.08328/) (2023)
- [GPT-4 and Safety Case Generation: An Exploratory Analysis](https://mlsanity.com/paper/2312.05696/) (2023)
- [Towards Engineering Fair and Equitable Software Systems for Managing Low-Altitude Airspace Authorizations](https://mlsanity.com/paper/2401.07353/) (2024)

## Built on by (most cited first, 5 of 5)

- [An LLM-Integrated Framework for Completion, Management, and Tracing of STPA](https://mlsanity.com/paper/2503.12043/) (2025)
- [A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models](https://mlsanity.com/paper/2511.00527/) (2025)
- [Safety Case Patterns for VLA-based driving systems: Insights from SimLingo](https://mlsanity.com/paper/2603.16013/) (2026)
- [From Safety Documentation to Safety Knowledge Support: An Evidence-Grounded LLM Framework for Medical Devices](https://mlsanity.com/paper/2608.12025/) (2026)
- [An Agentic RAG and Evaluation Framework for Assurance Case Generation: Industrial Use Case for the EU Cyber Resilience Act Compliance](https://mlsanity.com/paper/2608.19509/) (2026)

## Abstract (arXiv)

Constructing assurance cases is a widely used, and sometimes required, process toward demonstrating that safety-critical systems will operate safely in their planned environment. To mitigate the risk of errors and missing edge cases, the concept of defeaters - arguments or evidence that challenge claims in an assurance case - has been introduced. Defeaters can provide timely detection of weaknesses in the arguments, prompting further investigation and timely mitigations. However, capturing defeaters relies on expert judgment, experience, and creativity and must be done iteratively due to evolving requirements and regulations. This paper proposes CoDefeater, an automated process to leverage large language models (LLMs) for finding defeaters. Initial results on two systems show that LLMs can efficiently find known and unforeseen feasible defeaters to support safety analysts in enhancing the completeness and confidence of assurance cases.