ECCV 2020 Daily - Wednesday

2 Spotlight Presentation 14 Anita Rau is a PhD student at UCL in London and a software engineer research intern at Niantic. Her work attempts to answer the question: To what extent are two images picturing the same 3D surfaces? She speaks to us ahead of her inaugural ECCV spotlight presentation today (Wednesday). Anita’s work proposes the idea of image box embeddings , borrowed from word box embeddings for natural language processing . It shows that an asymmetric measure in world space can be used to interpret the relationship between two images and tell you if you are looking at the same thing. The method approximates the visible surface overlap between images and models the relative camera pose more intuitively than other methods, which require looking at the rotation and translation differences. It uses zoom-in and zoom-out relationships to predict the relative scale difference between two images. “If you know that one image is a zoom- in of another one and you want to do feature matching between the two, it is really useful to know the scale difference,” Anita tells us. “If you use SIFT features you have to search in scale space as well. With our method, you would know the scale already.” As you can see in the presentation video, the model uses neural nets to embed images into boxes. This is a new idea for representing the relationship between camera poses and has the potential to be used in many applications, such as image-based rendering and navigation, 3D scene reconstruction, robot navigation, and relocalization. Finding usable data was a challenge for the team, because they needed to know camera poses and the depths of pixels for each image, and there are not many datasets available with this information. In the end, they used a dataset for depth prediction and that worked out well. In the future, Anita would like to use different kinds of data to overcome the need for expensive 3D data and depth maps. For example, using homographies Predicting Visual Overlap of Images Through Interpretable Non-Metric Box Embeddings DAILY W e d n e s d a y