Computer Vision News - May 2016

COMPUTER VISION NEWS 18 Research The novelties of the presented approach are: (1) real-time RGB method with no depth data required; (2) mouth regions synthesized exclusively from the target sequence (no need for a teeth proxy); (3) state-of-the-art results, outperforming other methods both in terms of resulting video quality and runtime; (4) method which can also be used for face tracking with outstanding results, though it is not the main focus of this work. 2. Per frame tracing: tracing the expression, pose (rotation, translation) and illumination of the source and target actor’s video. These parameters are estimated for every frame with a dense analysis-by-synthesis statistical approach. 3. Expression Transfer: re-render the target’s face using the video parameters calculated in the previous steps. This re-rendering transformation expression function also preserved the estimated environment lighting of the target video. 4. Mouth Retrieval: synthesize a realistic target mouth region by retrieving and warping the best matching mouth image from the target actor sequence. This warping approach leads to much more realistic results than previous methods, which either copy the source mouth region onto the target or render a generic teeth proxy. This unique step is illustrated in the figure below. “ Accurate tracking, appearance, and lighting estimation in unconstrained live RGB video ”