Computer Vision News 4 and classification models. These models struggled because simulations oversimplified real-world cases with unrealistic textures and lighting conditions. Ruyu calls this the Appearance Gap. Modern deep generative models are optimized for realism, so it should not be an issue. However, there is still a noticeable performance drop when training. She subsequently identified two further gaps: the Content Gap and the Quality Gap. The Content Gap is a gap in the attributes of the generated data compared to the real-world dataset. “The most obvious gap is at the class level,” Ruyu points out. “Your model fails to generate a class of data in your dataset. Nowadays, powerful generative models don’t make these kinds of stupid mistakes, but they can make a second mistake, which is more likely to be overlooked. You don’t have a missing class, but for objectcentered images, some specific attributes might be missing because some combinations are rare.” For example, a typical cat image will be classified as a cat, but if the cat is wearing a Christmas costume, it might be dropped or barely generated by the model, as it is much rarer. Even if the generative model learns it, straightforward sampling may struggle to capture it, leading to a distribution mismatch between the sampled and training datasets. Lastly, the Quality Gap relates to anomalies in the deep generative data – for instance, semantic artifacts like a two-headed cat. In this paper, Ruyu conducts several investigations to pinpoint the reasons behind the domain gap between synthetic-to-real data. Her first experiment WACV Poster Presentation
RkJQdWJsaXNoZXIy NTc3NzU=