mlsanityData to 13 Oct 2026

Specificity-Aware Diffusion Steering via Variance-Reduced Sequential Monte Carlo

arXiv 2610.00395 ↗First posted 30 Sep 2026cs.LGcited by 0

On the map: Diffusion models → Diffusion foundations and sampling also fits: Flow matching, time series and science

Abstract (arXiv)

Inference-time steering enables pretrained diffusion models to satisfy new constraints without full retraining. However, specificity-aware generation is difficult: repelling samples from a negative reference distribution can also erode the positive distribution where the two overlap. The key challenge is to suppress negative mass while minimally distorting the positive distribution. We address this problem by formulating specificity-aware steering as a target-design problem and deriving a target distribution from an overlap-based objective. The resulting target keeps the desired reference distribution only in regions where it is sufficiently preferred over the undesired reference distribution, giving a likelihood-ratio interpretation of specificity. To sample from the corresponding time-dependent target path, we develop a Sequential Monte Carlo sampler with a variance-minimized local proposal. We further introduce a practical fixed-noise optimization procedure with the Jacobian--vector products with the desired and undesired score fields. Experiments on synthetic task, class-contrastive generation, text-to-image tasks and peptide-MHC (p-MHC) binder show that the proposed method suppresses undesired regions more effectively, reduces mode shift, and improves sampling stability by decreasing the SMC weight collapse compared with negative-guidance baselines. Code is available at: https://github.com/WangLuran/Specificity-Aware-Diffusion-Steering

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 32 of its 43 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

32 of 43 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Models25 Sep
  2. 2025RNE: plug-and-play diffusion inference-time control and energy-based training6 Jun
  3. 2025Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Design28 May
  4. 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts4 Mar
  5. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review16 Jan
  6. 2025Test-time Alignment of Diffusion Models without Reward Over-optimization10 Jan
  7. 2024Dynamic Negative Guidance of Diffusion Models18 Oct
  8. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models12 Jun
  9. 2022An optimal control perspective on diffusion-based generative modeling2 Nov
  10. 2022Classifier-Free Diffusion Guidance26 Jul
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
4 more
  1. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Implicit Models6 Oct
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  3. 2022Flow Matching for Generative Modelingcited by 5,542
  4. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 928
  5. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 620
  6. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 509
  7. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
  8. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  9. 2019Learning with Fenchel-Young Lossescited by 118
  10. 2019Elements of Sequential Monte Carlocited by 64
  11. 2025Training-Free Safe Denoisers for Safe Use of Diffusion Modelscited by 20
  12. 2025Psi-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Modelscited by 15
4 more
  1. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Modelscited by 9
  2. 2025Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimizationcited by 7
  3. 2020Minimizing relative entropy of path measures under marginal constraintscited by 4
  4. 2026ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedulecited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Models
  2. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Models
  3. 2026Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance
  4. 2026Tempered Guided Diffusion
  5. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction
  6. 2026A Plug-in Interpretation of Conditioning in Score-Based Diffusion Models
  7. 2026Robust Inference-Time Steering of Protein Diffusion Models via Embedding Optimization
  8. 2026Variance Reduction for Expectations with Diffusion Teachers