mlsanityData to 14 Oct 2026

MOSAIC: Multi-Objective Slice-Aware Iterative Curation for Alignment

arXiv 2603.18637 ↗First posted 19 Mar 2026cs.CR cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

We study how to allocate a fixed supervised fine-tuning budget when three objectives must be balanced at once: multi-turn safety alignment, low over-refusal on benign boundary queries, and instruction following under verifiable constraints. We propose MOSAIC (Multi-Objective Slice-Aware Iterative Curation for Alignment), a multi-objective framework for closed-loop data mixture search built on a unified L1-L3 evaluation interface. MOSAIC turns slice-level failure profiles into executable data actions, including dataset-level mixture ratios, bucket-level weights, and focus criteria. Under a fixed 1M-token budget and five rounds of independent fine-tuning from the same base model, MOSAIC improves internal XGuard from 2.76 to 4.67 while keeping OrBench at 4.41 and IFEval at 3.65. The final Pareto solution also generalizes better than a random static LoRA baseline on independent attack, over-refusal, and capability tests, suggesting that structured failure diagnosis can serve as a practical control signal for budgeted data construction. Code is available at https://github.com/douyipu/mosaic.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 16 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

11 of 16 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
  2. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct

Other references · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  6. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  7. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  8. 2024CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklistscited by 94
  9. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Continual Safety Alignment via Gradient-Based Sample Selection
  2. 2026GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training
  3. 2026DataShield: Uncovering Risky Fine-Tuning Data Across LLMs Through Consensus Subspace Alignment
  4. 2026MOSAIC: Composable Safety Alignment with Modular Control Tokens
  5. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignment
  6. 2026Safety-Utility Conflicts Are Not Global: Surgical Alignment via Head-Level Diagnosis
  7. 2026Data-Efficient Curation for Multimodal Reasoning under Fixed Training Protocols
  8. 2025Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning