Key points
- Netflix Eyeline Studios research enables free-viewpoint facial relighting from flat-lit performance captures.
- Combines a subject-specific diffusion model with dynamic Gaussian splatting reconstruction.
- Reproduces eye reflections, subsurface scattering and self-shadowing previously requiring manual VFX work.
Summary
DifFRelight, presented at SIGGRAPH Asia 2024 by Netflix's Eyeline Studios, introduces a diffusion-based system for relighting facial performance captures from any viewpoint without on-set lighting rigs for each required condition. The approach combines a subject-specific diffusion model with dynamic 3D Gaussian splatting to reconstruct the subject, then applies learned relighting. The result reproduces physically accurate lighting phenomena -- eye reflections, subsurface scattering, self-shadowing -- that previously required manual VFX. This represents a paradigm shift in the digital-human capture pipeline that educators teaching facial performance and visual effects need to understand as it enters production practice.
Related items
- Foundry CopyCat ML on Dune: Part Two (Production Case Study)
- The Artist-Driven Innovation Behind the Films We Love
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering
Source
Source: Eyeline Studios (Netflix) ↗ (Studio Statement)
Cite this item
Eyeline Studios (Netflix) (2024). ‘DifFRelight: Diffusion-Based Facial Performance Relighting’, Eyeline Studios (Netflix). Available at: https://www.eyelinestudios.com/research/diffrelight.html
Your reference manager can also read this page directly: with the Zotero (or Mendeley) browser connector installed, save it straight to your library. Whole-collection exports: RIS, BibTeX, CSL-JSON.