Key points
- Plug in motion module animates any personalised Stable Diffusion model without retraining.
- Separates appearance from motion, enabling character consistent generated animation.
- Foundation for thousands of node based animation workflows used in teaching.
Summary
AnimateDiff, accepted as a spotlight at ICLR 2024, introduces a plug-in motion module that can animate any personalised Stable Diffusion model without requiring model-specific retraining. By separating appearance (handled by the base model) from motion (handled by the shared motion module), it enables character-consistent generated animation with a wide range of visual styles. The framework became the entry-point architecture for AI video in animation curricula and underpins thousands of node-based workflows used in teaching labs globally.
Related items
- Adding Conditional Control to Text-to-Image Diffusion Models (ControlNet)
- ToonCrafter: Generative Cartoon Interpolation
- Wan: Open and Advanced Large-Scale Video Generative Models
- Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
- LoRA: Low-Rank Adaptation of Large Language Models
Source
Source: ICLR 2024 (spotlight) ↗ (Research)
Cite this item
Guo, Y., Yang, C., Rao, A., Liang, Z., Wang, Y., Qiao, Y., Agrawala, M., Lin, D. & Dai, B. (2024). ‘AnimateDiff: Animate Your Personalized Text-to-Image Models without Specific Tuning’, ICLR 2024 (spotlight). Available at: https://arxiv.org/abs/2307.04725
Your reference manager can also read this page directly: with the Zotero (or Mendeley) browser connector installed, save it straight to your library. Whole-collection exports: RIS, BibTeX, CSL-JSON.