PhotoFramer: Multi-modal Image Composition Instruction
Not on the map. Why?
Abstract (arXiv)
Composition matters during the photo-taking process, yet many casual users struggle to frame well-composed images. To provide composition guidance, we introduce PhotoFramer, a multi-modal composition instruction framework. Given a poorly composed image, PhotoFramer first describes how to improve the composition in natural language and then generates a well-composed example image. To train such a model, we curate a large-scale dataset. Inspired by how humans take photos, we organize composition guidance into a hierarchy of sub-tasks: shift, zoom-in, and view-change tasks. Shift and zoom-in data are sampled from existing cropping datasets, while view-change data are obtained via a two-stage pipeline. First, we sample pairs with varying viewpoints from multi-view datasets, and train a degradation model to transform well-composed photos into poorly composed ones. Second, we apply this degradation model to expert-taken photos to synthesize poor images to form training pairs. Using this dataset, we finetune a model that jointly processes and generates both text and images, enabling actionable textual guidance with illustrative examples. Extensive experiments demonstrate that textual instructions effectively steer image composition, and coupling them with exemplars yields consistent improvements over exemplar-only baselines. PhotoFramer offers a practical step toward composition assistants that make expert photographic priors accessible to everyday users.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 50 of its 78 references that are in the map. First posted on arXiv 30 Nov 2025.
Builds on
50 of 78 references.
References · 50
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
38 more
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
- 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 177
- 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 139
- 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Modelscited by 104
- 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
- 2025VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rankcited by 77
- 2021Mobile Computational Photography: A Tourcited by 62
- 2025ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understandingcited by 36
- 2025Harnessing Diffusion-Yielded Score Priors for Image Restorationcited by 34
- 2019Reliable and Efficient Image Cropping: A Grid Anchor based Approachcited by 32
- 2017Quantitative Analysis of Automatic Image Cropping Algorithms: A Dataset and Comparative Studycited by 26
- 2019Grid Anchor based Image Cropping: A New Benchmark and An Efficient Modelcited by 25
- 2017Learning to Compose with Professional Photographs on the Webcited by 24
- 2021Image Composition Assessment with Saliency-augmented Multi-pattern Poolingcited by 21
- 2019Image Cropping with Composition and Saliency Aware Aesthetic Score Mapcited by 20
- 2017Deep Cropping via Attention Box Prediction and Aesthetics Assessmentcited by 16
- 2024Spatial-Semantic Collaborative Cropping for User Generated Contentcited by 12
- 2017Automatic Image Cropping for Visual Aesthetic Enhancement Using Deep Neural Networks and Cascaded Regressioncited by 11
- 2023Learning Subject-Aware Cropping by Outpainting Professional Photoscited by 11
- 2024Instruction-based Image Manipulation by Watching How Things Movecited by 11
- 2022Human-centric Image Cropping with Partition-aware and Content-preserving Featurescited by 10
- 2023Beyond Image Borders: Learning Feature Extrapolation for Unbounded Image Compositioncited by 9
- 2021AutoPhoto: Aesthetic Photo Capture using Reinforcement Learningcited by 7
- 2025ProCrop: Learning Aesthetic Image Cropping from Professional Compositionscited by 7
- 2025Parametric Shadow Control for Portrait Generation in Text-to-Image Diffusion Modelscited by 5
- 2025Towards Smart Point-and-Shoot Photographycited by 5
- 2025Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimizationcited by 5
- 2018CAPTAIN: Comprehensive Composition Assistance for Photo Takingcited by 2
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026LumiVideo: An Intelligent Agentic System for Video Color Gradingcited by 1
- 2026AesFormer: Transform Everyday Photos into Beautiful Memoriescited by 1
- 2026CROP: Expert-Aligned Image Cropping via Compositional Reasoning and Optimizing Preferencecited by 0
- 2026ShotCrop3: Cropping Human-Centric Images into Cinematic Triple-Shot Compositionscited by 0
- 2026COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Modelscited by 0
- 2026Learning visual representations for compositional analysis of artworks and photographscited by 0
- 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Productioncited by 0
- 2026GeoComposer: Geometry-Grounded Photographic Composition Instructioncited by 0
- 2026GeoComposer: Geometry-Grounded Photographic Composition Instruction22 Sep 2026
- 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Production30 Aug 2026
- 2026Learning visual representations for compositional analysis of artworks and photographs6 Aug 2026
- 2026COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models27 Jun 2026
- 2026ShotCrop3: Cropping Human-Centric Images into Cinematic Triple-Shot Compositions4 Jun 2026
- 2026AesFormer: Transform Everyday Photos into Beautiful Memories21 May 2026
- 2026CROP: Expert-Aligned Image Cropping via Compositional Reasoning and Optimizing Preference9 May 2026
- 2026LumiVideo: An Intelligent Agentic System for Video Color Grading2 Apr 2026