mlsanityData to 14 Oct 2026

ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation

arXiv 2503.22194 ↗First posted 28 Mar 2025cs.CV cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce ORIGEN, the first zero-shot method for 3D orientation grounding in text-to-image generation across multiple objects and diverse categories. While previous work on spatial grounding in image generation has mainly focused on 2D positioning, it lacks control over 3D orientation. To address this, we propose a reward-guided sampling approach using a pretrained discriminative model for 3D orientation estimation and a one-step text-to-image generative flow model. While gradient-ascent-based optimization is a natural choice for reward-based guidance, it struggles to maintain image realism. Instead, we adopt a sampling-based approach using Langevin dynamics, which extends gradient ascent by simply injecting random noise--requiring just a single additional line of code. Additionally, we introduce adaptive time rescaling based on the reward function to accelerate convergence. Our experiments show that ORIGEN outperforms both training-based and test-time guidance methods across quantitative metrics and user studies.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 71 of its 89 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

71 of 89 references.

References · 71

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023Segment Anythingcited by 8,234
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  10. 2023Consistency Modelscited by 1,822
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
38 more
  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  2. 2024Depth Anything V2cited by 1,422
  3. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  4. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  5. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  6. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  8. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  9. 2023Adversarial Diffusion Distillationcited by 759
  10. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  11. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  12. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  13. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  14. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  15. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  16. 2023Universal Guidance for Diffusion Modelscited by 437
  17. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 395
  18. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  19. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  20. 2022Perception Prioritized Training of Diffusion Modelscited by 303
  21. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 294
  22. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 288
  23. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 266
  24. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  25. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  26. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  27. 2023Scalable 3D Captioning with Pretrained Modelscited by 203
  28. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  29. 2023Manifold Preserving Guided Diffusioncited by 157
  30. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 143
  31. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimizationcited by 123
  32. 2024D-Flow: Differentiating through Flows for Controlled Generationcited by 122
  33. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 119
  34. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 112
  35. 2023End-to-End Diffusion Latent Optimization Improves Classifier Guidancecited by 108
  36. 2024TFG: Unified Training-Free Guidance for Diffusion Modelscited by 107
  37. 2018Segmentation-driven 6D Object Pose Estimationcited by 104
  38. 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Modelscited by 95

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulationcited by 9
  2. 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
  3. 2026RewardFlow: Generate Images by Optimizing What You Rewardcited by 3
  4. 2025CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Explorationcited by 2
  5. 2026Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenescited by 2
  6. 2026Gradient Preconditioning for Efficient and Reliable Reward-Guided Generationcited by 2
  7. 2026Gradient-Free Noise Optimization for Reward Alignment in Generative Modelscited by 1
  8. 2026Thinking in Boxes: 3D Editing in Real Images Made Easycited by 0
  9. 2026PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenescited by 0
  10. 2026Manifold-Constrained Initial Noise Optimization for Efficient Generative Model Alignmentcited by 0