ECCV 2018 Daily - Monday

Daily MONDAY 10 Alexandra Carlson and Katherine Skinner are PhD candidates at the University of Michigan. They speak to us about the poster they presented yesterday as part of the workshop on Visual Learning and Embodied Agents in Simulation Environments (VLEASE). Alexandra tells us that the work is looking at how in general we can reduce the domain gap between synthetic and real data. The motivation for the work is looking at the failure modes of object detection networks and noticing that a lot of failure modes occur in incidences of overexposure or saturation. They hypothesised that modelling camera effects or sensor effects within images can help reduce the domain gap between synthetic and real data . Alexandra explains: “ By taking physically based models of how the camera degrades images – specifically chromatic aberration, blur, exposure, sensor noise, and colour balance – we’ve been able to demonstrate that each of these sensor effects is quite important in incorporating in the image augmentation process, in comparison to standard and often used augmentations .” Katherine adds that something she finds really interesting about the work is that when they augment with their augmentation parameters for sensor effects, they can actually outperform training on a much larger dataset without their augmentation . That hints that augmenting in the sensor domain is just as important as augmenting in a spatial domain, or environmental effects or other effects. Surprisingly, the work was almost very different, as Alexandra reveals: “ One of the things that we initially tried was learning the parameter distributions in the sensor effect domain from real data. The evening before our intended deadline submission we realised that domain randomisation over the sensor domains was better. So, just four hours before the deadline, we had to change the story of the paper! ” Thinking about next steps for the work, Alexandra says that their current model is purely image augmentation method, and so they are looking at developing modules that can be inserted into deep neural networks that will learn how to perform this augmentation during the training process. Modeling Camera Effects to Improve Visual Learning from Synthetic Data 10 Presentation