Key points
- DreamFusion introduces Score Distillation Sampling, which uses gradients from a pretrained 2D diffusion model to optimise a 3D NeRF representation from a text prompt, without any 3D training data.
- It received the ICLR 2023 Outstanding Paper award and seeded the Magic3D, ProlificDreamer and DreamCraft3D lineage.
- DreamFusion's Score Distillation Sampling loss established the text-to-3D category and underlies a generation of asset-generation tools entering animation and game-art pipelines.
Summary
Poole et al. (2022, published ICLR 2023) present DreamFusion, the first method to generate 3D assets from text descriptions by distilling knowledge from a pretrained 2D diffusion model into a NeRF via a novel Score Distillation Sampling loss. The approach requires no 3D training data and generalises to arbitrary text prompts. Recognised as an ICLR 2023 Outstanding Paper, DreamFusion established the text-to-3D category and its Score Distillation Sampling technique directly underlies subsequent tools entering animation and game-art production pipelines.
Related items
- CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets
- High-Resolution Image Synthesis with Latent Diffusion Models
Source
Source: ICLR 2023 ↗ (Research)
Cite this item
Poole, B., Jain, A., Barron, J. T. & Mildenhall, B. (2023). ‘DreamFusion: Text-to-3D Using 2D Diffusion’, ICLR 2023. Available at: https://arxiv.org/abs/2209.14988
Your reference manager can also read this page directly: with the Zotero (or Mendeley) browser connector installed, save it straight to your library. Whole-collection exports: RIS, BibTeX, CSL-JSON.