Computer Vision News - December 2019

Research 4 Every month, Computer Vision News reviews a research paper from our field. This month we have chosen SinGAN: Learning a Generative Model from a Single Natural Image. We are indebted to the authors (Tamar Rott Shaham, Tali Dekel, Tomer Michaeli), for allowing us to use their images. This paper won the Best Paper Award at ICCV 2019, last month in Seoul. by Amnon Geifman In the last few years, two subfields of neural nets have developed: the first is internal learning which is a learning paradigm that exploits the internal distribution of patches inside a single image to perform several computer vision tasks. The second is GANs- generative adversarial networks which, as we all know, is a framework that mimics a distribution of a given dataset by training a generator that fools a discriminator by generating good looking images. Today's paper incorporates these two fields to formulate a novel framework that trains a generator from a single image. This generator learns to capture the internal distribution of patches in a single image by training on different scales of the same image, against patch discriminator. This enables the model, at test time, to generate high quality diverse samples based on the original image. The paper demonstrates the applicability of the framework to many computer vision tasks such as super resolution, harmonization, editing, paint to image and more. SinGAN Model The goal of the model is to learn a generator that captures that internal distribution of patches in an image. To this end, the authors suggest a generative framework consisting of a hierarchy of patch-GAN, each applied on a different scale of the image. Specifically, the model consists of a pyramid of generators {G 0 ,G 1 ,..,G N } : each generator is trained against a different scale of the input image {x 0 ,x 1 ,..,x N }, where x n is the downsampled version of the image x by a scale factor r n >1. At every scale, G n learns to generate an image-sample in which all overlapping patches cannot be distinguished from patches in the original down sampled image x n . Best of ICCV 2019