Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
Detecting abnormal events in real-world customer service dialogues is highly challenging due to the complexity of business data and the dynamic nature of customer interactions. Moreover, models must demonstrate strong out-of-domain (OOD) generalization to enable rapid adaptation across different business scenarios and maximize commercial value. In this work, we propose a novel Adaptive Perplexity-Aware Reinforcement Learning (APARL) framework that leverages the advanced reasoning capabilities of large language models for abnormal event detection. APARL introduces a dual-loop dynamic curriculum learning architecture, enabling the model to progressively focus on more challenging samples as its proficiency increases. This design effectively addresses performance bottlenecks and significantly enhances OOD transferability. Extensive evaluations on food delivery dialogue tasks show that our model achieves significantly enhanced adaptability and robustness, attaining the highest F1 score with an average improvement of 17.19%, and an average improvement of 9.59% in OOD transfer tests. This method provides a superior solution for industrial deployment of anomaly detection models, contributing to improved operational efficiency and commercial benefits.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 63 references that are in the map. First posted on arXiv 2 Jul 2025.
Builds on
38 of 63 references.
Closest ancestors · 10
Same direction, newest first
- 2025RM-R1: Reward Modeling as Reasoning5 May
- 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models13 Apr
- 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
- 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
- 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
- 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
- 2017Proximal Policy Optimization Algorithms20 Jul
Other references · 28
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
16 more
- 2024OpenAI o1 System Cardcited by 1,244
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
- 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 187
- 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Modelscited by 144
- 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 108
- 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 83
- 2023ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Applicationcited by 48
- 2019Ray Interference: a Source of Plateaus in Deep Reinforcement Learningcited by 40
- 2025QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generationcited by 35
- 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
- 2025o3-mini vs DeepSeek-R1: Which One is Safer?cited by 24
- 2021Multi-class Text Classification using BERT-based Active Learningcited by 11
- 2025Token-Hungry, Yet Precise: DeepSeek R1 Highlights the Need for Multi-Step Reasoning Over Speed in MATHcited by 10
- 2023Fine-tuned vs. Prompt-tuned Supervised Representations: Which Better Account for Brain Language Representations?cited by 8
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2026Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Textcited by 13
- 2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learningcited by 11
- 2025PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modelingcited by 6
- 2026Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text15 Jan 2026
- 2025PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling28 Oct 2025
- 2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning13 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection
- 2025Benchmarking and Learning Real-World Customer Service Dialogue
- 2025AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization
- 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning
- 2025PATS: Process-Level Adaptive Thinking Mode Switching
- 2025Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
- 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures
- 2026ChatAD: Reasoning-Enhanced Time-Series Anomaly Detection with Multi-Turn Instruction Evolution