Key points
- pix2pix established supervised paired image-to-image translation with a conditional GAN and patch discriminator.
- CycleGAN solved the unpaired case via cycle consistency, enabling photo-to-anime and style-domain transfer without paired data.
- Together they underlie sketch colourisation, edge-to-image and concept-to-frame pipelines, and the paired-training idea behind ControlNet.
Summary
Isola et al. (pix2pix, CVPR 2017) and Zhu et al. (CycleGAN, ICCV 2017) established the two canonical approaches to image-to-image translation. pix2pix trains a conditional GAN on aligned image pairs to learn a mapping between visual domains, such as edge maps to photographs or sketches to coloured artwork. CycleGAN removes the paired-data requirement by enforcing cycle consistency, learning to translate between domains from unaligned collections and enabling applications such as photo-to-anime and style-domain transfer. Together these papers underlie sketch colourisation, edge-to-image generation and concept-to-frame pipelines in animation production, and they are the paired-training ancestor that ControlNet explicitly cites. Both are foundational content in AI and animation courses.
Related items
- Adding Conditional Control to Text-to-Image Diffusion Models (ControlNet)
- Generative Adversarial Networks (GAN)
Source
Source: CVPR 2017 / ICCV 2017 (IEEE/CVF) ↗ (Research)
Cite this item
Isola, P., Zhu, J.-Y., Zhou, T. & Efros, A. A. (2017). ‘Image-to-Image Translation: pix2pix and CycleGAN’, CVPR 2017 / ICCV 2017 (IEEE/CVF). Available at: https://arxiv.org/abs/1611.07004
Your reference manager can also read this page directly: with the Zotero (or Mendeley) browser connector installed, save it straight to your library. Whole-collection exports: RIS, BibTeX, CSL-JSON.