mlsanityData to 7 Oct 2026

BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversion

arXiv 2408.04785 ↗First posted 8 Aug 2024cs.CVcited by 1

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Textual Inversion remains a popular method for personalizing diffusion models, in order to teach models new subjects and styles. We note that textual inversion has been underexplored using alternatives to the UNet, and experiment with textual inversion with a vision transformer. We also seek to optimize textual inversion using a strategy that does not require explicit use of the UNet and its idiosyncratic layers, so we add bonus tokens and enforce orthogonality. We find the use of the bonus token improves adherence to the source images and the use of the vision transformer improves adherence to the prompt. Code is available at https://github.com/jamesBaker361/tex_inv_plus.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 32 of its 51 references that are in the map. First posted on arXiv 8 Aug 2024.

Builds on

32 of 51 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning23 Apr
  2. 2024BootPIG: Bootstrapping Zero-shot Personalized Image Generation Capabilities in Pretrained Diffusion Models25 Jan
  3. 2023An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesis20 Nov
  4. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Models16 Nov
  5. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  6. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning21 Jul
  7. 2023Real-World Image Variation by Aligning Diffusion Inversion Chain30 May
  8. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models25 May
  9. 2022DreamArtist++: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter21 Nov
  10. 2022Diffusion-based Image Translation using Disentangled Style and Content Representation30 Sep
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Generative Adversarial Networkscited by 5,603
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2014Conditional Generative Adversarial Netscited by 2,152
9 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  2. 2016Generative Adversarial Text to Image Synthesiscited by 735
  3. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  4. 2015A Neural Algorithm of Artistic Stylecited by 480
  5. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  6. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  7. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  8. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  9. 2024MuseumMaker: Continual Style Customization without Catastrophic Forgettingcited by 10

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20

Similar, not cited

Close in wording, no citation either way

  1. 2023P+: Extended Textual Conditioning in Text-to-Image Generation
  2. 2026Dual Inversion for Text-to-Image Diffusion Models: From Both Prompt and Noise Perspectives
  3. 2025Training-Free Diffusion Priors for Text-to-Image Generation via Optimization-based Visual Inversion
  4. 2022Direct Inversion: Optimization-Free Text-Driven Real Image Editing with Diffusion Models
  5. 2025Controllable 3D Object Generation with Single Image Prompt
  6. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models
  7. 2026Zero-Shot Personalization of Objects via Textual Inversion
  8. 2026TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformers