# LogiDebrief: A Signal-Temporal Logic based Automated Debriefing Approach with Large Language Models Integration

arXiv: https://arxiv.org/abs/2505.03985 · HTML: https://mlsanity.com/paper/2505.03985/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 30 of 30)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack](https://mlsanity.com/paper/2406.10149/) (2024)
- [SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning](https://mlsanity.com/paper/2308.00436/) (2023)
- [Deductive Verification of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2306.03872/) (2023)
- [Searching for Best Practices in Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01219/) (2024)
- [Can Large Language Models Reason and Plan?](https://mlsanity.com/paper/2403.04121/) (2024)
- [A Comparative Study on Reasoning Patterns of OpenAI's o1 Model](https://mlsanity.com/paper/2410.13639/) (2024)
- [Why Does the Effective Context Length of LLMs Fall Short?](https://mlsanity.com/paper/2410.18745/) (2024)
- [nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models](https://mlsanity.com/paper/2303.04864/) (2023)
- [When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1](https://mlsanity.com/paper/2410.01792/) (2024)
- [Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation](https://mlsanity.com/paper/2406.12114/) (2024)
- [Exploring Context Window of Large Language Models via Decomposed Positional Vectors](https://mlsanity.com/paper/2405.18009/) (2024)
- [Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning](https://mlsanity.com/paper/2311.17365/) (2023)
- [Mastering Symbolic Operations: Augmenting Language Models with Compiled Neural Networks](https://mlsanity.com/paper/2304.01665/) (2023)
- [CitySpec: An Intelligent Assistant System for Requirement Specification in Smart Cities](https://mlsanity.com/paper/2206.03132/) (2022)
- [CitySpec with Shield: A Secure Intelligent Assistant for Requirement Formalization](https://mlsanity.com/paper/2302.09665/) (2023)
- [Predictive Monitoring with Logic-Calibrated Uncertainty for Cyber-Physical Systems](https://mlsanity.com/paper/2011.00384/) (2020)
- [Sim911: Towards Effective and Equitable 9-1-1 Dispatcher Training with an LLM-Enabled Simulation](https://mlsanity.com/paper/2412.16844/) (2024)
- [An Intelligent Assistant for Converting City Requirements to Formal Specification](https://mlsanity.com/paper/2206.07152/) (2022)
- [Auto311: A Confidence-guided Automated System for Non-emergency Calls](https://mlsanity.com/paper/2312.14185/) (2023)
- [Combining LLMs with Logic-Based Framework to Explain MCTS](https://mlsanity.com/paper/2505.00610/) (2025)

## Built on by (most cited first, 5 of 5)

- [Counterfactual Fairness Evaluation of LLM-Based Contact Center Agent Quality Assurance System](https://mlsanity.com/paper/2602.14970/) (2026)
- [Learning with Preserving for Continual Multitask Learning](https://mlsanity.com/paper/2511.11676/) (2025)
- [Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learned](https://mlsanity.com/paper/2602.13241/) (2026)
- [PACE: A Personalized Adaptive Curriculum Engine for 9-1-1 Call-taker Training](https://mlsanity.com/paper/2603.05361/) (2026)
- [Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming](https://mlsanity.com/paper/2607.23019/) (2026)

## Abstract (arXiv)

Emergency response services are critical to public safety, with 9-1-1 call-takers playing a key role in ensuring timely and effective emergency operations. To ensure call-taking performance consistency, quality assurance is implemented to evaluate and refine call-takers' skillsets. However, traditional human-led evaluations struggle with high call volumes, leading to low coverage and delayed assessments. We introduce LogiDebrief, an AI-driven framework that automates traditional 9-1-1 call debriefing by integrating Signal-Temporal Logic (STL) with Large Language Models (LLMs) for fully-covered rigorous performance evaluation. LogiDebrief formalizes call-taking requirements as logical specifications, enabling systematic assessment of 9-1-1 calls against procedural guidelines. It employs a three-step verification process: (1) contextual understanding to identify responder types, incident classifications, and critical conditions; (2) STL-based runtime checking with LLM integration to ensure compliance; and (3) automated aggregation of results into quality assurance reports. Beyond its technical contributions, LogiDebrief has demonstrated real-world impact. Successfully deployed at Metro Nashville Department of Emergency Communications, it has assisted in debriefing 1,701 real-world calls, saving 311.85 hours of active engagement. Empirical evaluation with real-world data confirms its accuracy, while a case study and extensive user study highlight its effectiveness in enhancing call-taking performance.