mlsanityData to 7 Oct 2026

A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic Learning Stick

arXiv 2406.15352 ↗First posted 21 Jun 2024cs.CLcited by 10

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Keyword mnemonics are memorable explanations that link new terms to simpler keywords. Prior work generates mnemonics for students, but they do not train models using mnemonics students prefer and aid learning. We build SMART, a mnemonic generator trained on feedback from real students learning new terms. To train SMART, we first fine-tune LLaMA-2 on a curated set of user-written mnemonics. We then use LLM alignment to enhance SMART: we deploy mnemonics generated by SMART in a flashcard app to find preferences on mnemonics students favor. We gather 2684 preferences from 45 students across two types: expressed (inferred from ratings) and observed (inferred from student learning), yielding three key findings. First, expressed and observed preferences disagree; what students think is helpful does not always capture what is truly helpful. Second, Bayesian models can synthesize complementary data from multiple preference types into a single effectiveness signal. SMART is tuned via Direct Preference Optimization on this signal, which resolves ties and missing labels in the typical method of pairwise comparisons, augmenting data for LLM output quality gains. Third, mnemonic experts assess SMART as matching GPT-4 at much lower deployment costs, showing the utility of capturing diverse student feedback to align LLMs in education.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 33 of its 90 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

33 of 90 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Aligning Language Models with Human Preferences via a Bayesian Approach9 Oct
  2. 2023LIMA: Less Is More for Alignment18 May

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  12. 2023Large Language Models are not Fair Evaluatorscited by 1,050
19 more
  1. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  3. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  4. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  5. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  6. 2023Poisoning Language Models During Instruction Tuningcited by 254
  7. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
  8. 2024Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scorescited by 50
  9. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  10. 2024Bayesian Reward Models for LLM Alignmentcited by 35
  11. 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 28
  12. 2022The Authenticity Gap in Human Evaluationcited by 24
  13. 2023Reinforcement Learning from Diverse Human Preferencescited by 24
  14. 2022Investigations of Performance and Bias in Human-AI Teamwork in Hiringcited by 16
  15. 2021Understanding Jargon: Combining Extraction and Generation for Definition Modelingcited by 15
  16. 2023SmartPhone: Exploring Keyword Mnemonic with Auto-generated Verbal and Visual Cuescited by 4
  17. 2019Combining Sentiment Lexica with a Multi-View Variational Autoencodercited by 2
  18. 2021PSG: Prompt-based Sequence Generation for Acronym Extractioncited by 2
  19. 2024KARL: Knowledge-Aware Retrieval and Representations aid Retention and Learning in Studentscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 2.

10 papers.

  1. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  2. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 25
  3. 2025Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personascited by 17
  4. 2025A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Userscited by 6
  5. 2024Exploring Automated Keyword Mnemonics Generation with Large Language Models via Overgenerate-and-Rankcited by 4
  6. 2025MODS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collectionscited by 4
  7. 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Userscited by 2
  8. 2025PhoniTale: Phonologically Grounded Mnemonic Generation for Typologically Distant Language Pairscited by 1
  9. 2026Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Stylecited by 1
  10. 2025Interpretable Mnemonic Generation for Kanji Learning via Expectation-Maximizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions
  2. 2024A Systematic Examination of Preference Learning through the Lens of Instruction-Following
  3. 2025String Seed of Thought: Prompting LLMs for Distribution-Faithful and Diverse Generation
  4. 2024Improving the Validity of Automatically Generated Feedback via Reinforcement Learning
  5. 2024Can LLMs Identify Gaps and Misconceptions in Students' Code Explanations?
  6. 2024Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback
  7. 2026Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident
  8. 2025Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents