Diversifying Detail and Appearance in Sketch-Based Face Image Synthesis

Abstract:

Sketch-based face image synthesis has gained greater attention with the increasing realism of its output images. However, existing studies have overlooked the significance of output diversity: because sketches are inherently ambiguous, it would be desirable to have various output candidates for a single input sketch. In this paper, we explore synthesis of diverse face images from a single sketch by using a three-stage framework consisting of sketch refinement, detail enhancement, and appearance synthesis. Each stage uses supervised learning with neural networks. With this threestage framework, we can separately control the detail (e.g., wrinkles and hair structures) and appearance (e.g., skin and hair colors) of output face images separately by using multiple latent codes. Quantitative and quantitative evaluations demonstrate that our method offers greater diversity in its output images than the state-of-the-art methods, while retaining the output realism.

Keywords: Sketch-based image synthesis; Deep learning; GAN; Multimodal

Video:

Publication:

1. Takato Yoshikawa, Yuki Endo, Yoshihiro Kanamori: "Diversifying Detail and Appearance in Sketch-Based Face Image Synthesis" The Visual Computer (Proc. of Computer Graphics Internatinal 2022), 2022. [PDF (2MB)][Code]

BibTeX Citation

@article{YoshikawaCGI22,
      author    = {Takato Yoshikawa and Yuki Endo and Yoshihiro Kanamori},
      title     = {Diversifying Detail and Appearance in Sketch-Based Face Image Synthesis},
      journal   = {The Visual Computer (Proc. of Computer Graphics Internatinal 2022)},
      volume    = {38},
      pages     = {3121--3133},
      year      = {2022}
    }

Last modified: Jun 2022