mlsanityData to 14 Oct 2026

RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination

arXiv 2505.21925 ↗First posted 28 May 2025cs.GR cs.CV cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

We present RenderFormer, a neural rendering pipeline that directly renders an image from a triangle-based representation of a scene with full global illumination effects and that does not require per-scene training or fine-tuning. Instead of taking a physics-centric approach to rendering, we formulate rendering as a sequence-to-sequence transformation where a sequence of tokens representing triangles with reflectance properties is converted to a sequence of output tokens representing small patches of pixels. RenderFormer follows a two stage pipeline: a view-independent stage that models triangle-to-triangle light transport, and a view-dependent stage that transforms a token representing a bundle of rays to the corresponding pixel values guided by the triangle-sequence from the view-independent stage. Both stages are based on the transformer architecture and are learned with minimal prior constraints. We demonstrate and evaluate RenderFormer on scenes with varying complexity in shape and light transport.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

31 of 54 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  10. 2020GLU Variants Improve Transformercited by 1,804
  11. 2021Vision Transformers for Dense Predictioncited by 1,803
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
19 more
  1. 2019Root Mean Square Layer Normalizationcited by 1,480
  2. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  3. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  4. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  5. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  6. 2021Advances in Neural Renderingcited by 365
  7. 2020Query-Key Normalization for Transformerscited by 280
  8. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
  9. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 189
  10. 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 130
  11. 2022Is Attention All That NeRF Needs?cited by 113
  12. 2021Real-time Neural Radiance Caching for Path Tracingcited by 91
  13. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  14. 2023A Survey on Long Text Modeling with Transformerscited by 47
  15. 2018RenderNet: A deep convolutional network for differentiable rendering from 3D shapescited by 42
  16. 2021Neural Radiositycited by 33
  17. 2023ReTR: Modeling Rendering Via Transformer for Generalizable Neural Surface Reconstructioncited by 30
  18. 2019Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Toolcited by 20
  19. 2020Learning Neural Light Transportcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 18.

23 papers.

Sort:

By area:

  1. 2026Generative World Renderercited by 10
  2. 2026Programmable World Modelcited by 10
  3. 2025CUPID: Generative 3D Reconstruction via Joint Object and Pose Modelingcited by 8
  4. 2025NeAR: Coupled Neural Asset-Renderer Stackcited by 5
  5. 2026RigidFormer: Learning Rigid Dynamics using Transformerscited by 3
  6. 2026WorldParticle: Unified World Simulation of Lagrangian Particle Dynamics via Transformercited by 3
  7. 2026ClothTransformer: Unified Latent-Space Transformers for Scalable Cloth Simulationcited by 3
  8. 2025A Generalizable Light Transport 3D Embedding for Global Illuminationcited by 2
  9. 2026RenderFlow: Single-Step Neural Rendering via Flow Matchingcited by 2
  10. 2026RnG: A Unified Transformer for Complete 3D Modeling from Partial Observationscited by 2
  11. 2026PhysiFormer: Learning to Simulate Mechanics in World Spacecited by 2
  12. 2025Neural Cone Radiosity for Interactive Global Illumination with Glossy Materialscited by 1
11 more
  1. 2026SPRig: Self-Supervised Pose-Invariant Rigging from Mesh Sequencescited by 1
  2. 2025From Particles to Fields: Reframing Photon Mapping with Continuous Gaussian Photon Fieldscited by 0
  3. 2026A Few-Step Generative Model on Cumulative Flow Mapscited by 0
  4. 2026TOPOS: High-Fidelity and Efficient Industry-Grade 3D Head Generationcited by 0
  5. 2026Learning Laplacian Eigenspace with Mass-Aware Neural Operators on Point Cloudscited by 0
  6. 2026Learning View-Dependent Splatting Kernelscited by 0
  7. 2026Physically-guided Image Generation for Multi-Projection Mappingcited by 0
  8. 2026RenderFormer++: Scalable and Physics-Informed Feed-Forward Neural Renderingcited by 0
  9. 2026ViDS: Video Diffusion Shader using 3D Face Trackingcited by 0
  10. 2026RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitivescited by 0
  11. 2026Physically Based Rendering in the Latent Spacecited by 0