mlsanityData to 7 Oct 2026

HelloMeme: Integrating Spatial Knitting Attentions to Embed High-Level and Fidelity-Rich Conditions in Diffusion Models

arXiv 2410.22901 ↗First posted 30 Oct 2024cs.CVcited by 6

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We propose an effective method for inserting adapters into text-to-image foundation models, which enables the execution of complex downstream tasks while preserving the generalization ability of the base model. The core idea of this method is to optimize the attention mechanism related to 2D feature maps, which enhances the performance of the adapter. This approach was validated on the task of meme video generation and achieved significant results. We hope this work can provide insights for post-training tasks of large text-to-image models. Additionally, as this method demonstrates good compatibility with SD1.5 derivative models, it holds certain value for the open-source community. Therefore, we will release the related code (urlhttps://songkey.github.io/hellomeme).

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 21 of its 26 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

21 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug

Other references · 18

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  7. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  8. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  9. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  10. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  11. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  12. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
6 more
  1. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
  2. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
  3. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
  4. 2023Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flowcited by 123
  5. 2022Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FCcited by 73
  6. 2024Tunnel Try-on: Excavating Spatial-temporal Tunnels for High-quality Virtual Try-on in Videoscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025AvatarShield: Visual Reinforcement Learning for Human-Centric Synthetic Video Detectioncited by 6
  2. 2025PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysiscited by 5
  3. 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representationscited by 2
  4. 2025Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Datacited by 1
  5. 2026The Alpha Blending Hypothesis: Compositing Shortcut in Deepfake Detectioncited by 0
  6. 2026CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models
  2. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
  3. 2023R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation
  4. 2024MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models
  5. 2023LLM-grounded Video Diffusion Models
  6. 2025Pushing the Boundaries of State Space Models for Image and Video Generation
  7. 2025Precise Parameter Localization for Textual Generation in Diffusion Models
  8. 2024Enhancing Image Layout Control with Loss-Guided Diffusion Models