ECCV 2020 Daily - Monday

2 Poster Presentation 14 DAILY M o n d a y Kripasindhu Sarkar is a postdoctoral researcher at the Graphics, Vision & VideogroupattheMaxPlanckInstitute for Informatics in Saarbrücken, Germany, led by Professor Christian Theobalt. He completed his PhD last year at DFKI Kaiserslautern.‬He speaks to us ahead of his poster today about neural re-rendering of humans‬.‬ Given a single input image, this work presents a method of re-rendering a high-quality image of a person with a target pose, viewpoint or garment . It is a new way of rendering called neural rendering, which uses a neural network. It is a fusion between computer graphics and computer vision that doesn’t need the explicit rendering models of computer graphics, like ray tracing or rasterization . Instead, it uses a simple computer vision feedforward neural network . The work has many practical applications in the fashion industry . Currently, retail websites need to put models in hundreds of different clothes and poses requiring many images. Using this method, the system can be trained on a limited number of images of the models wearing some specific garments, and other clothes and poses can be transferred from elsewhere. As you can see in the presentation video, this method represents the appearance of the model in a high-dimensional UV feature map instead of a normal colored texture map. The feature map is learned implicitly end to end, which gives high- quality results. Kripasindhu tells us more about how the work uses the tools of computer vision: “We are using a convolutional neural network and we represent the human model with a probabilistic model called SMPL . We do the final rendering using a U-Net-like architecture which is also taken from computer vision.” The work uses an existing training and testing dataset called DeepFashion , which provides around 50,000 images of fashion models in different clothes, although Kripasindhu is quick to point out that his method does not need a lot of training data . It trains with pairs of Neural Re-Rendering of Humans from a Single Image