Facilitating Human Feedback for GenAI Prompt Optimization
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Abstract (arXiv)
This study investigates the optimization of Generative AI (GenAI) systems through human feedback, focusing on how varying feedback mechanisms influence the quality of GenAI outputs. We devised a Human-AI training loop where 32 students, divided into two groups, evaluated AI-generated responses based on a single prompt. One group assessed a single output, while the other compared two outputs. Preliminary results from this small-scale experiment suggest that comparative feedback might encourage more nuanced evaluations, highlighting the potential for improved human-AI collaboration in prompt optimization. Future research with larger samples is recommended to validate these findings and further explore effective feedback strategies for GenAI systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 0 of its 6 references that are in the map. First posted on arXiv 29 Mar 2024.
Builds on
0 of 6 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Automating Creativity
- 2026Human-Machine Collaboration on Generative Meta-Learning: Model and Algorithm
- 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
- 2026Evaluating adaptive and generative AI-based feedback and recommendations in a knowledge-graph-integrated programming learning system
- 2026QQJ: Quantifying Qualitative Judgment for Scalable and Human-Aligned Evaluation of Generative AI
- 2024Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
- 2025Generative to Agentic AI: Survey, Conceptualization, and Challenges
- 2024Generative Reward Models