mlsanityData to 14 Oct 2026

Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems

arXiv 2406.18245 ↗First posted 26 Jun 2024cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

The inherent ambiguity of cause and effect boundaries poses a challenge in evaluating causal event extraction tasks. Traditional metrics like Exact Match and BertScore poorly reflect model performance, so we trained evaluation models to approximate human evaluation, achieving high agreement. We used them to perform Reinforcement Learning with extraction models to align them with human preference, prioritising semantic understanding. We successfully explored our approach through multiple datasets, including transferring an evaluator trained on one dataset to another as a way to decrease the reliance on human-annotated data. In that vein, we also propose a weak-to-strong supervision method that uses a fraction of the annotated data to train an evaluation model while still achieving high performance in training an RL model. Our code is available at https://github.com/oyarsa/event_extraction/tree/causal-event-extraction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 27 of its 43 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

27 of 43 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  12. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
15 more
  1. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
  2. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  3. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 119
  4. 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
  5. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  6. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 72
  7. 2023RewriteLM: An Instruction-Tuned Large Language Model for Text Rewritingcited by 67
  8. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  9. 2023An Empirical Study on Information Extraction using Large Language Modelscited by 46
  10. 2023Retrieval-Augmented Code Generation for Universal Information Extractioncited by 40
  11. 2020Financial Document Causality Detection Shared Task (FinCausal 2020)cited by 31
  12. 2021Survey on reinforcement learning for language processingcited by 31
  13. 2019Causality Extraction based on Self-Attentive BiLSTM-CRF with Transferred Embeddingscited by 25
  14. 2022Robust Preference Learning for Storytelling via Contrastive Reinforcement Learningcited by 14
  15. 2023Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extractioncited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Reward Modeling with Weak Supervision for Language Models
  2. 2026EAGER: Enhancing Generative Event Extraction via Reinforcement Learning with Verifiable Rewards
  3. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
  4. 2026CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
  5. 2024Semi-Supervised Reward Modeling via Iterative Self-Training
  6. 2025Preference Learning for AI Alignment: a Causal Perspective
  7. 2025Reward Models are Metrics in a Trench Coat
  8. 2025SAFER: Probing Safety in Reward Models with Sparse Autoencoder